Bulk Drive Stream Retrieval
The mean time for a successful retrieval of 10 images from a Drive stream is around 30 seconds. The mean time for specific geographic regions can vary. Retrieval can fail for a variety of reasons and optimizing how the API is used is essential to maximizing image yield. Some common strategies are discussed below.
Efficiently Retrieving a Large Number of Drives
When retrieving drives for large timespans and/or large geographic areas, it will be necessary to retrieve the drives in multiple pages since the limit is 10,000 drives per query. The "offset" parameter is the simplest way to retrieve multiple pages. However large offset values can result in slower retrieval times.
One alternative technique that can be used is to set the "after" parameter for the next query to be the time of the most recent drive returned by the previous query. In this way, offset can remain at zero and retrieval times are faster.
Efficient Use of Concurrent Requests
Each client is assigned a maximum number of concurrent requests to the API. Concurrent requests are the total number of requests which have started, but not finished, at any given time. Note that an API request for 10 images counts as 10 concurrent requests. For most applications there will be many more Drives that match desired locations than there are concurrent requests and time to retrieve the images. For this reason, it’s inefficient to dedicate one concurrent request to retrieving Drive data. Instead, occasionally retrieve Drive data and store any matches to desired locations, and then allocate all concurrent requests to image retrieval.
Note: Multiple concurrent requests can be used to retrieve Drive data faster. For example, rather than use a single concurrent request to retrieve 15 minutes (900 seconds) of data, use 10 concurrent requests to retrieve 90 seconds of data each.
Avoiding Drives That Miss a Location
The most reliable method to find drives that pass through a desired location is to simply look for Drives that start or end very near to the location. However this may not produce enough matches in rural areas, in which case all drives in the vicinity need to be considered as possible matches. The Drives Through a Point sample describes techniques that can be used in this case.
Avoiding Failed Requests
To minimize requests for drive streams from a vehicle that goes offline, request Drive data at least every 15 minutes, and only request drive streams from a VehicleId that was online in the most recently retrieved data.
Attempting to collect multiple Drives from the same vehicle concurrently may increase the likelihood of timeouts. Keep an “in progress” list of vehicle Ids and avoid using the same vehicle concurrently.
An on-vehicle camera that fails to return a Drive stream is very likely to fail a subsequent request. After a failure, keep the vehicle Id on a “failed” list for 15 minutes. Also ensure the camera is online before making more requests.
Avoiding Unusable Images
From some applications, images from nighttime or twilight are less usable than daytime images, either from lack of ambient light or blurring caused by longer camera exposures. Use the latitude, longitude and time of a Drive to determine whether the Drive stream is from the daytime. NOAA has example code for solar calculations.
When an on-vehicle camera faces directly into the sun, the image can be overexposed in the areas of interest. Calculate the sun altitude and azimuth, and avoid the combination of low altitude with an azimuth that is similar to the Drive heading.
Partial Drive Stream Retrieval
For some applications, only a few images are required from a full Drive Stream. Rather than retrieving the full stream, predict where in the drive the region of interest falls, and use the “offset” parameter to retrieve the 10 closest images. For example, if the region of interest is predicted to be about halfway through a 60 second drive, set the offset to 25 to retrieve the 26th to 35th images.