Decimal separator REST-API: clarification needed

[kristofpoppe]

We're testing the rest-api and are trying to figure out in what format the answers of decimal values are presented.

In our current VB0365 setup (most recent version) , we have some locale settings on the windows machine Belgian Comma, which uses a comma as decimal separator. Are then the values stores also in a comma format ? Or are these values locale-independent ?

This is necessary to find out how we can and will parse the values from the API for further usage in a Grafana dashboard.

Thanks for any input and/or clarification.

[nielsengelen]

They are normally locale independent. There is already a Grafana dashboard available, have a look at https://github.com/VeeamHub/automation/ ... 65-grafana

[kristofpoppe]

Hi Niels, thanks for your prompt response. We are indeed testing with this .sh script, but when analysing the data we get from the server we see comma's.

So we altered the script.: awk -F. '{print $1}') changed to separate on a comma instead of a point, but then we receive
unable to parse..... invalid boolean.

Also when we look at the JSON, the comma's are already there where we expect points, neither if we change the "accepted language" in the curl command.

[nielsengelen]

Is the bash script failing? If so can you try using: awk -F',' (with quotes as this could be important)? Or are you getting the parse error somewhere else?

[kristofpoppe]

Indeed, the bash gives now errors on parsing the data.
Below a short sample of the output. For your information, we replaced on all places the awk -F','

HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 5451e386-18d9-11e9-8482-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.7.3
X-Request-Id: 5451e386-18d9-11e9-8482-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT

HTTP/1.1 400 Bad Request
Content-Type: application/json
Request-Id: 545a6154-18d9-11e9-8483-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Error: unable to parse 'veeam_office365_restoresession,organization=All organizations,veeamjobname=organizations,type=Vex,result=Success,initiatedBy=VEEAM4OFF365\Administrator itemsProcessed=0,itemsSuccess=0 1532929781': invalid boolean
X-Influxdb-Version: 1.7.3
X-Request-Id: 545a6154-18d9-11e9-8483-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT
Content-Length: 243

{"error":"unable to parse 'veeam_office365_restoresession,organization=All organizations,veeamjobname=organizations,type=Vex,result=Success,initiatedBy=VEEAM4OFF365\\Administrator itemsProcessed=0,itemsSuccess=0 1532929781': invalid boolean"}
HTTP/1.1 204 No Content
Content-Type: application/json
Request-Id: 54625eb4-18d9-11e9-8484-005056ac9cfe
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.7.3
X-Request-Id: 54625eb4-18d9-11e9-8484-005056ac9cfe
Date: Tue, 15 Jan 2019 15:22:11 GMT

[nielsengelen]

I've asked Jorge to jump on here as I don't know the full bash script. It may be needed to rework the parameters with some code so they change the "," into a "." before parsing them.

[jorgedlcruz]

Good morning guys,
Sorry for the delay, if you try the swagger on your browser https://YOURVBOIP:4443/swagger/ui, what values do you see under Job Sessions?

I see the next ones:


I will check the code today and see if I can get rid of that awk anyways using floats when we write back to InfluxDB.

[wishr]

Hi gentlemen,

I'd like to add that our QA team has checked several locales (on both server and client) and confirmed that regardless of a locale set they were getting responses with the "." (dot) separator in decimal values. Most likely that happens due to "JSON features" - in JSON commas are used for separating key-value pairs, while dots are used for decimal values.

Hope this helps. Thanks.

[jorgedlcruz]

Hello,
Please try the next script, I've changed the way we retrieve the number:

• https://github.com/jorgedlcruz/veeam-ba ... 65-grafana

Basically the RESTfulAPI always gives you . which is what I am using now as well to write back to InfluxDB, as it will not accept commas as a value.

In case for whatever reason you want to obtain a comma for the decimals, you can find the change now on the next part of the code:

Code: Select all

awk -F'[^0-9]*' '{print $1 "." $2}'

So you can put your , there, however that will not work for InfluxDB, so I will let it as it is.

Please let us know, also let us know if you have any trouble with the Grafana Dashboard itself, it is working fine for me.

[kristofpoppe]

Hi Jorge, many thanks for your input.

In the meanwhile we've tested it with swagger and below is the output: (indeed comma's)
We'll test your updated script and give feedback. We're also figuring out where the parsing of the boolean goes wrong.

[kristofpoppe]

We're thinking the parser will fail on the GET /v2/RestoreSessions where we have "0 items processed"


"initiatedBy": "VEEAM4OFF365\\Administrator",
"details": "0 items processed",
"_links": {

[jorgedlcruz]

Okay, so this is a different problem then, this is on the last part of the Script, has the previous problem been solved?

[jorgedlcruz]

I can not find any result like that, for me on my RestoreSessions I have the next:

Code: Select all

{
  "offset": 0,
  "limit": 30,
  "results": [
    {
      "id": "2584e02b-7da2-49d2-9d3f-02150ee4c26f",
      "name": "Veeam Explorer for Exchange (Job: BJ-EXCHANGE-ALL)",
      "organization": "jorgedelacruz.onmicrosoft.com",
      "type": "Vex",
      "creationTime": "2018-12-15T20:13:25.1245205Z",
      "endTime": "2018-12-15T20:15:24.339536Z",
      "state": "Stopped",
      "result": "Success",
      "initiatedBy": "ZIMBRA\\Administrator",
      "details": "16 items processed (16 successes)",
      "_links": {
        "self": {
          "href": "https://192.168.1.33:4443/v2/restoresessions/2584e02b-7da2-49d2-9d3f-02150ee4c26f"
        },
        "organization": {
          "href": "https://192.168.1.33:4443/v2/restoresessions/2584e02b-7da2-49d2-9d3f-02150ee4c26f/organization"
        }
      }
    },

I can try to improve the script in case I had 0 restore sessions but I will need from you that you paste the entire JSON you receive. As said, I think the main problem you had at the start of the thread it is now solved?

[jorgedlcruz]

Ha! I just saw your reply with the commas, kristofpoppe, that's fine, the new code should work for that, please let me know about the RestoreSessions, so we can fix it too in case it fails

[wishr]

Hi Kristof,

I'd like to jump in as well and ask you what locale do you have installed on the server and what are the locale settings in use. We'd like to perform an internal analysis of that.

Thanks

[kristofpoppe]

Update:

Also with the new code, we're still receiving comma's but we were able to 'fix' this

So we added extra ‘missing’ awk for readrate, because there were still comma’s. This is now good interpreted, but also “readrate=” sometimes has no values.
Perhaps this explains the “missing field value”-error ? The only error which now persists is “’invalid Boolean”, so to which variable should this map?
Perhaps related to the inconsistent use of backslashes in the value of “initiatedBy”?

Another thought: we’re running veeam-stats.ps1 on the same linux box, which also uses rest-api and that’s working perfect.

[Polina]

Hi Kristof,

Our engineers are on this issue, but it will take some time to confirm and troubleshoot. Stay tuned for updates.

[Polina]

Some feedback from our REST API guru:

First, it’s not a JSON issue, but the specifics of the way the server sends data. The values you see under Job Session statistics are the decimal units converted by the server itself from “raw” bytes (and this is where comma comes from). The decimals and their corresponding units of measurement (MB/GB/TB) are then sent as a string, which may cause parsing issues.

This behavior will be improved in the next releases.

Thanks!

[b.vanhaastrecht]

I also stumbled on this, looking forward on being fixed.