Google Play Metadata

Starting December 2023, we provide our collection of Google Play Metadata.
Note: The metadata is only for Google Play.
We started acquiring Metadata in May 2020, directly from the (independent, incomplete) protobuf implementation of the protocol used by the official Google Play app to talk to the Google Play services.
Some elements of metadata are related to an app (ex: 'com.chrome.canary'), some others are related specifically to one version of an app.
Not all metadata are obtained close to the release date. It is frequent in this dataset to find new metadata for old versions, for example: metadata obtained in 2023 for a version released in 2020. In that case, the app-related elements will reflect information at the date the metadata was acquired (2023), not information at the date of release of this version (2020). As such, the number of downloads (among other fields) in the metadata will be that of 2023, not of 2020.

We include a field az_metadata_date that contains the date we acquired this piece of metadata. This is the only modification made to the metadata.
If you use these metadata, please cite the following paper:

@INPROCEEDINGS{10555779,
  author={Alecci, Marco and Ruiz Jiménez, Pedro Jesús and Allix, Kevin and Bissyandé, Tegawendé F. and Klein, Jacques},
  booktitle={2024 IEEE/ACM 21st International Conference on Mining Software Repositories (MSR)}, 
  title={AndroZoo: A Retrospective with a Glimpse into the Future}, 
  year={2024},
  volume={},
  number={},
  pages={389-393},
  keywords={Aggregates;Refining;Software;Reproducibility of results;Internet;Data mining;Android Applications;APK;Software Repository;Metadata},
  doi={}}

Elements related to an app (and hence: to the date of collection):

• descriptionHtml
• descriptionShort
• details.appDetails.numDownloads
• details.appDetails.recentChangesHtml
• details.appDetails.versionCode
• aggregateRating (and everything inside)
• And propably others...

Elements related to a specific version:

• details.appDetails.versionCode
• details.appDetail.installationSize
• details.appDetail.uploadDate

Weekly-generated files

• gp-metadata-full.jsonl.gz: Contains each and every record we have, in jsonlines (i.e., each line is a valid JSON). There can be several records for one app. There can be several records for one (app, version) couple.
  As described above, records may have been acquired in an order, and at a time that does not follow the release of new versions.
  Correspondig APKs may or may not be in AndroZoo.
  7.1 GB compresed (as of December 2023)
• gp-metadata-aggregate.jsonl.gz: Contains one line per app.
  1.2 GB compresed (as of December 2023)
  This file is generated to ease searches through metadata. With this file, it's trivial to filter apps based on starRating, numDownloads, etc.
  Note: This file is easily usable with jq
  Example of a JSON object in that file:

  {
    "pkg_name": "com.fiverr.fiverr",
    "nb_meta": 219, // Number of metadata records in AndroZoo
    "nb_versionCode": 170,  // Those records talk about 170 different versions
    "min_versionCode": 350,  // The minimum versionCode we have metadata for
    "max_versionCode": 535,  // The maximum versionCode we have metadata for
    "first_seen": "2020-05-17 06:05:35.027723", // the oldest metadata record
    "last_seen": "2023-12-05 21:17:02.214049",  // the newest metadata record
    "min_star_rating": "4.139585",
    "max_star_rating": "5.0",
    "min_ratingsCount": 167541,
    "max_ratingsCount": 317214,
    "min_commentCount": 221,
    "max_commentCount": 140520,
    "min_upload_date": "2020-05-04", // the oldest (normalised) details.appDetail.uploadDate field in all records for this app
    "max_upload_date": "2023-12-03", // the newest (normalised) details.appDetail.uploadDate field in all records for this app
    "min_nb_downloads": 10000000, // The minimum (normalised) value for the field details.appDetails.numDownloads
    "max_nb_downloads": 10000000,
    "min_installationSize": 23633468,
    "max_installationSize": 48212052,
    "developerName": [ // The list of all values for the field developerName
      "Fiverr"
    ],
    "developerEmail": [ // The list of all values for the field developerEmail
      "support@fiverr.com"
    ],
    "developerWebsite": [ // The list of all values for the field developerWebsite
      "http://www.fiverr.com"
    ],
    "developerAddress": [ // The list of all values for the field developerAddress
      ""
    ],
    "related_apks_in_AZ_info": { // Information about related APKs that are in AndroZoo
      "nb_apks": 346, // might be zero
      "first_apk_az_date": "2014-03-14 14:56:16.637949", // oldest date one version (APK) of this app was added to AndroZoo
      "last_apk_az_date": "2023-12-05 23:15:20.986874",
      "nb_versionCode": 323,  // in AndroZoo, we have APKs for 323 different versionCodes
      "min_versionCode": 4,  // Note that we have versions from much before we started to collect metadata
      "max_versionCode": 535,
      "min_vt_detection": 0, 
      "max_vt_detection": 1, // at most 1 AntiVirus detected at least one version as potentially harmful
      "min_size": 8758304,
      "max_size": 48212052,
      "sha256s": [ // might be empty
        "7DEB60DBE1CB761A49330419F2FA91FE0F3C8339C5B90185EBB1D7096A64FC6E",
          // A whole lot more sha256s corresponding to APKs that are available in AndroZoo
      ]
    }
  }

Those two files are only distributed to registered users of AndroZoo.
To obtain them:
curl -O --remote-header-name -G -d apikey=${APIKEY} 'https://androzoo.uni.lu/api/get_gp_metadata_file/full'
curl -O --remote-header-name -G -d apikey=${APIKEY} 'https://androzoo.uni.lu/api/get_gp_metadata_file/aggregate'

API

• /api/get_gp_metadata/PKG_NAME and /api/get_gp_metadata/PKG_NAME/VERSIONCODE
  Replace PKG_NAME with the name of an app (ex: 'com.chrome.canary') to get a JSON list (NOT jsonLines) of all the metada records we have about this app (for all versions).
  From the command line: Example: curl -G -d apikey=${APIKEY} 'https://androzoo.uni.lu/api/get_gp_metadata/occam.hammer.drone'
  To limit to only versionCode 65: curl -G -d apikey=${APIKEY} 'https://androzoo.uni.lu/api/get_gp_metadata/occam.hammer.drone/65'

Limitations

Given the incompletness of our protobuf protocol definition, some fields have no name.

Example of one metadata record

{
    "aggregateRating": {
      "commentCount": "0",
      "fiveStarRatings": "0",
      "fourStarRatings": "0",
      "oneStarRatings": "0",
      "ratingsCount": "0",
      "starRating": 0.0,
      "threeStarRatings": "0",
      "twoStarRatings": "0",
      "type": 2
    },
    "availability": {
      "restriction": 1
    },
    "az_metadata_date": "2023-09-17 14:03:39.220798",  // This field is added by AndroZoo: date this record was added to AndroZoo. This is NOT related to the Google-provided 'uploadDate' below.
    "backendDocid": "occam.hammer.drone",  // This is the app name (pkg_name)
    "backendId": 3,
    "creator": "Hammer Missions",
    // the following field contains HTML (escaped for rendering here)
    "descriptionHtml": "Hammer provides *adaptive* mission planning for drones, enabling high-quality data collection for a number of use-cases.<br><br>Whatever the mission, Hammer&#39;s got you covered!<br><br>Hammer is the one-stop solution for planning and flying different types of drone flights automatically with high levels of precision and control. Say goodbye to constantly switching between apps or learning new systems all the time.<br><br>With our features, you can be sure that your flight planning is reliable and data capture on the field is quick and accurate. Hammer can operate in the connectionless mode - once a planned mission has been uploaded to the drone, no further connection is necessary to execute the mission.<br><br>*MISSIONS TYPES*<br><br>1. Inspection: Hammer supports all types of inspection missions - be it roof inspection or pipeline inspections. Simply select the horizontal or linear inspection option from the app&#39;s in-built mission store and you&#39;re good to go. Hammer also supports vertical inspection for surveying building facades / tower blocks / cliffs, etc.<br><br>2. Mapping: Hammer supports 2D/3D mapping using the standard grid-based flight plans. The double-grid option can also be used to create high precision 3D models. Hammer also supports linear and tower mapping, to create orthomosaic maps for railways, tracks, towers, etc.<br><br>3. Orbits &amp; Waypoints [COMING SOON]: Sometimes to get that perfect shot, you need higher levels of customisation. Hammer&#39;s waypoint mission can be used exactly for that. With curved turns mode, a smooth flight path can be generated to record cinematographic videos and cable-car shots.<br><br>*SUPPORTED DRONES*<br><br>- DJI Mavic Air / Mavic Pro, Mavic 2 Pro / Entreprise / Zoom<br>- DJI Inspire 2<br>- DJI Phantom 4<br>- DJI Matrice 200, 210<br><br>For a complete list, please visit: https://bit.ly/2ZtIEl2<br><br>*SUPPORTED CAMERAS*<br><br>- All fixed Cameras ( Spark, Mavic 2, Phantom 4 )<br>- X3<br>- X4S<br>- X5R<br>- X5S<br>- XT<br>- X7<br>- XT2Thermal<br>- XT2Visual<br><br> For more info:<br><br>- terms of use: https://hub.hammermissions.com/terms-of-service<br>- privacy policy: https://hub.hammermissions.com/privacy"
    "descriptionShort": "One Software. Many Missions. Quality Drone Data.",
    "details": {
      "appDetails": {
        "dependencies": {
          "dependency": [
            {
              "packageName": "com.google.android.gms",
              "version": 12451000
            }
          ],
          "unknown2": "117666285",
          "unknown3": 33
        },
        "developerEmail": "team@hammermissions.com",
        "developerName": "Hammer Missions",
        "developerWebsite": "https://hammermissions.com",
        "file": [
          {
            "fileType": 0,
            "size": "54244902",
            "versionCode": 65
          },
          {
            "fileType": 0,
            "size": "2759857",
            "versionCode": 65
          },
          {
            "fileType": 0,
            "size": "102809",
            "versionCode": 65
          },
          {
            "fileType": 0,
            "size": "60558717",
            "versionCode": 65
          }
        ],
        "installationSize": "117666285",
        "numDownloads": "500+ downloads",
        "packageName": "occam.hammer.drone",
        "permission": [
          "android.permission.ACCESS_COARSE_LOCATION",
          "android.permission.ACCESS_FINE_LOCATION",
          "android.permission.ACCESS_NETWORK_STATE",
          "android.permission.ACCESS_WIFI_STATE",
          "android.permission.BLUETOOTH",
          "android.permission.BLUETOOTH_ADMIN",
          "android.permission.CHANGE_NETWORK_STATE",
          "android.permission.CHANGE_WIFI_STATE",
          "android.permission.EXPAND_STATUS_BAR",
          "android.permission.FOREGROUND_SERVICE",
          "android.permission.GET_TASKS",
          "android.permission.INTERNET",
          "android.permission.KILL_BACKGROUND_PROCESSES",
          "android.permission.READ_EXTERNAL_STORAGE",
          "android.permission.READ_PHONE_STATE",
          "android.permission.RECEIVE_BOOT_COMPLETED",
          "android.permission.RECORD_AUDIO",
          "android.permission.SYSTEM_ALERT_WINDOW",
          "android.permission.VIBRATE",
          "android.permission.WAKE_LOCK",
          "android.permission.WRITE_EXTERNAL_STORAGE",
          "com.google.android.finsky.permission.BIND_GET_INSTALL_REFERRER_SERVICE",
          "com.google.android.gms.permission.AD_ID"
        ],
        // The folowing field contains HTML (escaped for rendering here)
        "recentChangesHtml": "Terrain Following*: Now make all your flights terrain aware with a few simple taps! Terrain Follow mode available for all mission types.<br>*Wind Turbine Inspections*: Introducing a new flight mode for Wind Turbine Inspections. Place your blade on the 12&#39;o position and inspect from all sides.<br>*Improved Camera Controls*: Improvements on drone camera controls.",
        "unstable": true,
        "uploadDate": "Aug 31, 2023",    // Beware of variations on the format of the date provided by Google
        "versionCode": 65,
        "versionString": "2.1.12"
      }
    },
    "detailsReusable": true,
    "detailsUrl": "details?doc=occam.hammer.drone",
    "docType": 1,
    "docid": "occam.hammer.drone",
    "image": [
      {
        "color": "#585153",
        "dimension": {
          "height": 500,
          "width": 1024
        },
        "imageType": 2,
        "imageUrl": "https://play-lh.googleusercontent.com/6sf8fTIiGqjfliLw_ut-_Q0lRErz_vDq8VGGVvbFzIHTw_5-q-8MOnVh2_1Li5dMNeQ",
        "supportsFifeUrlOptions": true
      },
      {
        "imageType": 3,
        "imageUrl": "https://www.youtube.com/watch?v=UwIupyZyxmc"
      },
      // other images removed from this example
    ],
    "offer": [
      {
        "checkoutFlowRequired": false,
        "currencyCode": "CAD",  //Most often set to the currency of the country of the crawler that collected this record. Sometimes, it seems random.
        "formattedAmount": "",
        "message": "Buy",
        "micros": "0",
        "offerType": 1
      }
    ],
    "relatedLinks": {
      "privacyPolicyUrl": "https://hub.hammermissions.com/privacy",
      "rated": {
        "image": {
          "dimension": {
            "height": 512,
            "width": 512
          },
          "imageType": 6,
          "imageUrl": "https://play-lh.googleusercontent.com/IciOnDFecb5Xt50Q2jlcNC0LPI7LEGxNojroo-s3AozcyS-vDCwtq4fn7u3wZmRna8OewG9PBrWC-i7i",
          "supportsFifeUrlOptions": true
        },
        "label": "Everyone",
        "learnMoreHtmlLink": "<a href="https://support.google.com/googleplay?p=appgame_ratings">Learn more</a>"
      }
    },
    "reviewQuestionsUrl": "reviewQuestions?doc=occam.hammer.drone",
    "reviewsUrl": "rev?doc=occam.hammer.drone&n=20",
      // The folowing field contains HTML (escaped for rendering here)
    "shareUrl": "https://play.google.com/store/apps/details?id=occam.hammer.drone",
    "title": "Hammer Missions",
    "unknown25": {
      "item": [
        {
          "container": {
            "value": "Hammer Missions"
          },
          "label": "Offered by"
        },
        {
          "container": {
            "value": "team@hammermissions.com"
          },
          "label": "Developer e-mail"
        },
        {
          "container": {
            "value": "May 26, 2022"
          },
          "label": "Released on"
        }
      ]
    },
    "unknownCategoryContainer": {}
  }