Creating JSON from iostat lines ; Adding timezone information to naive datetimes?

Victor Hooi victorhooi at gmail.com
Thu Jul 2 23:51:01 EDT 2015


I just want to run some things past you guys, to make sure I'm doing it right.

I'm using Python to parse disk metrics out of iostat output. The device lines look like this:

    Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
    sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

My goal is JSON output for each metric that look like the below (this is for InfluxDB):

  {
    "measurement": "read_requests",
    "tags": {
      "project": "SOME_PROJECT",
      "hostname": "server1.newyork.com",
    },
    "time": timestamp.isoformat(),
    "fields": {
      "value": 0.00
    }
  }

To create the above, I am using the following code:

  disk_stat_headers = ['device', 'read_requests_merged', 'write_requests_merged', 'read_requests', 'write_requests', 'read_sectors', 'write_sectors', 'average_request_size', 'average_queue_length', 'average_wait', 'average_service_time', 'utilisation']
  ...<code to iterate over lines>...
    elif i >= 5 and line:
      disk_stats = {}
      device = line.split()[0]
      disk_stats[device] = dict(zip(disk_stat_headers, line.split()[1:]))

      json_points = []
      for disk_name, metrics in disk_stats.items():
          print(disk_name)
          print(metrics)
          for key, value in metrics.items():
              json_points.append({
                  "measurement": key,
                  "tags": {
                      "project": project,
                      "hostname": hostname,
                  },
                  "time": timestamp.isoformat(),
                  "fields": {
                      "value": value
                  }
              })


Is there any issue with the above? Or can you see a better way to do this?

(I'm calling split() twice, not sure if that's worse than storing it in a variable)

Second question - the timestamps in isotat are timezone-naive. I'm using the below to add the right timezone (EDT in this case) to them, which depends on the pytz library:

  from pytz import timezone
  eastern = timezone('US/Eastern')
  timestamp = eastern.localize(line)

Is the above an appropriate way of doing this? Or is there an easier way just using the Python stdlib's?



More information about the Python-list mailing list