Maintain control of your Microsoft 365 data
Post Reply
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Decimal separator REST-API: clarification needed

Post by kristofpoppe »

We're testing the rest-api and are trying to figure out in what format the answers of decimal values are presented.

In our current VB0365 setup (most recent version) , we have some locale settings on the windows machine Belgian Comma, which uses a comma as decimal separator. Are then the values stores also in a comma format ? Or are these values locale-independent ?

This is necessary to find out how we can and will parse the values from the API for further usage in a Grafana dashboard.

Thanks for any input and/or clarification.
nielsengelen
Product Manager
Posts: 5618
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Decimal separator REST-API: clarification needed

Post by nielsengelen »

They are normally locale independent. There is already a Grafana dashboard available, have a look at https://github.com/VeeamHub/automation/ ... 65-grafana
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Re: Decimal separator REST-API: clarification needed

Post by kristofpoppe »

Hi Niels, thanks for your prompt response. We are indeed testing with this .sh script, but when analysing the data we get from the server we see comma's.

So we altered the script.: awk -F. '{print $1}') changed to separate on a comma instead of a point, but then we receive
unable to parse..... invalid boolean.

Also when we look at the JSON, the comma's are already there where we expect points, neither if we change the "accepted language" in the curl command.
nielsengelen
Product Manager
Posts: 5618
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Decimal separator REST-API: clarification needed

Post by nielsengelen »

Is the bash script failing? If so can you try using: awk -F',' (with quotes as this could be important)? Or are you getting the parse error somewhere else?
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Re: Decimal separator REST-API: clarification needed

Post by kristofpoppe »

Indeed, the bash gives now errors on parsing the data.
Below a short sample of the output. For your information, we replaced on all places the awk -F','

HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 5451e386-18d9-11e9-8482-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.7.3
X-Request-Id: 5451e386-18d9-11e9-8482-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT

HTTP/1.1 400 Bad Request
Content-Type: application/json
Request-Id: 545a6154-18d9-11e9-8483-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Error: unable to parse 'veeam_office365_restoresession,organization=All organizations,veeamjobname=organizations,type=Vex,result=Success,initiatedBy=VEEAM4OFF365\Administrator itemsProcessed=0,itemsSuccess=0 1532929781': invalid boolean
X-Influxdb-Version: 1.7.3
X-Request-Id: 545a6154-18d9-11e9-8483-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT
Content-Length: 243

{"error":"unable to parse 'veeam_office365_restoresession,organization=All organizations,veeamjobname=organizations,type=Vex,result=Success,initiatedBy=VEEAM4OFF365\\Administrator itemsProcessed=0,itemsSuccess=0 1532929781': invalid boolean"}
HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 54625eb4-18d9-11e9-8484-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.7.3
X-Request-Id: 54625eb4-18d9-11e9-8484-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT
nielsengelen
Product Manager
Posts: 5618
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Decimal separator REST-API: clarification needed

Post by nielsengelen »

I've asked Jorge to jump on here as I don't know the full bash script. It may be needed to rework the parameters with some code so they change the "," into a "." before parsing them.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
jorgedlcruz
Veeam Software
Posts: 1355
Liked: 613 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: Decimal separator REST-API: clarification needed

Post by jorgedlcruz »

Good morning guys,
Sorry for the delay, if you try the swagger on your browser https://YOURVBOIP:4443/swagger/ui, what values do you see under Job Sessions?

I see the next ones:
Image

I will check the code today and see if I can get rid of that awk anyways using floats when we write back to InfluxDB.
Jorge de la Cruz
Senior Product Manager | Veeam ONE @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2024 / InfluxAce / Grafana Champion
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Decimal separator REST-API: clarification needed

Post by wishr »

Hi gentlemen,

I'd like to add that our QA team has checked several locales (on both server and client) and confirmed that regardless of a locale set they were getting responses with the "." (dot) separator in decimal values. Most likely that happens due to "JSON features" - in JSON commas are used for separating key-value pairs, while dots are used for decimal values.

Hope this helps. Thanks.
jorgedlcruz
Veeam Software
Posts: 1355
Liked: 613 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: Decimal separator REST-API: clarification needed

Post by jorgedlcruz »

Hello,
Please try the next script, I've changed the way we retrieve the number: Basically the RESTfulAPI always gives you . which is what I am using now as well to write back to InfluxDB, as it will not accept commas as a value.

In case for whatever reason you want to obtain a comma for the decimals, you can find the change now on the next part of the code:

Code: Select all

awk -F'[^0-9]*' '{print $1 "." $2}'
So you can put your , there, however that will not work for InfluxDB, so I will let it as it is.

Please let us know, also let us know if you have any trouble with the Grafana Dashboard itself, it is working fine for me.
Jorge de la Cruz
Senior Product Manager | Veeam ONE @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2024 / InfluxAce / Grafana Champion
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Re: Decimal separator REST-API: clarification needed

Post by kristofpoppe » 1 person likes this post

Hi Jorge, many thanks for your input.

In the meanwhile we've tested it with swagger and below is the output: (indeed comma's)
We'll test your updated script and give feedback. We're also figuring out where the parsing of the boolean goes wrong.

Image
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Re: Decimal separator REST-API: clarification needed

Post by kristofpoppe »

We're thinking the parser will fail on the GET /v2/RestoreSessions where we have "0 items processed"


"initiatedBy": "VEEAM4OFF365\\Administrator",
"details": "0 items processed",
"_links": {
jorgedlcruz
Veeam Software
Posts: 1355
Liked: 613 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: Decimal separator REST-API: clarification needed

Post by jorgedlcruz »

Okay, so this is a different problem then, this is on the last part of the Script, has the previous problem been solved?
Jorge de la Cruz
Senior Product Manager | Veeam ONE @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2024 / InfluxAce / Grafana Champion
jorgedlcruz
Veeam Software
Posts: 1355
Liked: 613 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: Decimal separator REST-API: clarification needed

Post by jorgedlcruz »

I can not find any result like that, for me on my RestoreSessions I have the next:

Code: Select all

{
  "offset": 0,
  "limit": 30,
  "results": [
    {
      "id": "2584e02b-7da2-49d2-9d3f-02150ee4c26f",
      "name": "Veeam Explorer for Exchange (Job: BJ-EXCHANGE-ALL)",
      "organization": "jorgedelacruz.onmicrosoft.com",
      "type": "Vex",
      "creationTime": "2018-12-15T20:13:25.1245205Z",
      "endTime": "2018-12-15T20:15:24.339536Z",
      "state": "Stopped",
      "result": "Success",
      "initiatedBy": "ZIMBRA\\Administrator",
      "details": "16 items processed (16 successes)",
      "_links": {
        "self": {
          "href": "https://192.168.1.33:4443/v2/restoresessions/2584e02b-7da2-49d2-9d3f-02150ee4c26f"
        },
        "organization": {
          "href": "https://192.168.1.33:4443/v2/restoresessions/2584e02b-7da2-49d2-9d3f-02150ee4c26f/organization"
        }
      }
    },
I can try to improve the script in case I had 0 restore sessions but I will need from you that you paste the entire JSON you receive. As said, I think the main problem you had at the start of the thread it is now solved?
Jorge de la Cruz
Senior Product Manager | Veeam ONE @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2024 / InfluxAce / Grafana Champion
jorgedlcruz
Veeam Software
Posts: 1355
Liked: 613 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: Decimal separator REST-API: clarification needed

Post by jorgedlcruz »

Ha! I just saw your reply with the commas, kristofpoppe, that's fine, the new code should work for that, please let me know about the RestoreSessions, so we can fix it too in case it fails
Jorge de la Cruz
Senior Product Manager | Veeam ONE @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2024 / InfluxAce / Grafana Champion
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: Decimal separator REST-API: clarification needed

Post by wishr »

Hi Kristof,

I'd like to jump in as well and ask you what locale do you have installed on the server and what are the locale settings in use. We'd like to perform an internal analysis of that.

Thanks
kristofpoppe
Veeam Vanguard
Posts: 60
Liked: 7 times
Joined: May 18, 2012 1:19 pm
Full Name: Kristof Poppe
Contact:

Re: Decimal separator REST-API: clarification needed

Post by kristofpoppe »

Update:

Also with the new code, we're still receiving comma's but we were able to 'fix' this

So we added extra ‘missing’ awk for readrate, because there were still comma’s. This is now good interpreted, but also “readrate=” sometimes has no values.
Perhaps this explains the “missing field value”-error ? The only error which now persists is “’invalid Boolean”, so to which variable should this map?
Perhaps related to the inconsistent use of backslashes in the value of “initiatedBy”?

Another thought: we’re running veeam-stats.ps1 on the same linux box, which also uses rest-api and that’s working perfect.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Decimal separator REST-API: clarification needed

Post by Polina »

Hi Kristof,

Our engineers are on this issue, but it will take some time to confirm and troubleshoot. Stay tuned for updates.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Decimal separator REST-API: clarification needed

Post by Polina »

Some feedback from our REST API guru:

First, it’s not a JSON issue, but the specifics of the way the server sends data. The values you see under Job Session statistics are the decimal units converted by the server itself from “raw” bytes (and this is where comma comes from). The decimals and their corresponding units of measurement (MB/GB/TB) are then sent as a string, which may cause parsing issues.

This behavior will be improved in the next releases.

Thanks!
b.vanhaastrecht
Service Provider
Posts: 833
Liked: 154 times
Joined: Aug 26, 2013 7:46 am
Full Name: Bastiaan van Haastrecht
Location: The Netherlands
Contact:

Re: Decimal separator REST-API: clarification needed

Post by b.vanhaastrecht »

I also stumbled on this, looking forward on being fixed.
======================================================
Veeam ProPartner, Service Provider and a proud Veeam Legend
Post Reply

Who is online

Users browsing this forum: No registered users and 21 guests