Release Notes

✨ New:

• General speed improvements
• New and improved onboarding flow

🪲 Bug Fixes:

• Fixed bug that didn’t change the geo on the download list button
• Fixed bug preventing some filters from being “sticky” in between searches
• Fixed bug preventing geography profile filters from updating charts

No API changes this month!

🏗️ Permits

• Total permits: The dataset now contains 171M permits — an increase of 2M compared to the previous batch. With 8 new jurisdictions.
• first_seen_date coverage: ~150M permits have a first_seen_date

🧑🏼‍🔧 Contractors

• We now have over 3 million contractors in the dataset, with ~50K new records added this month
• License linkage: An additional 80K contractors have been linked to state license files.

📇 Classification coverage

• All contractors with a license now have the classification_derived field as well. This field maps state-specific classifications into standardized categories.
• If a contractor has multiple license classifications, they are concatenated using the pipe (”|”) character.

🗂️ License Files

• A new table, CSL, is added to our production database. It contains data from 26 states files, and has a total of 1.8 M contractors.
• Around 200K contractors are linked to our Contractors table while the others are new. As part of this new dataset, we have around 700K new contractors phone numbers and 200K new emails.

✨ New:

• Added Website Search filter options to Contractor Filters
• Added tooltips to the UI to better explain functionality

🚀 Upgrades:

• Added Search filter auto-caching to improve UX between page reloads and sessions
• Added new New User onboarding experience
• Adjusted datepicker widget for date range selection to monthly, which will improve usability

✨ New:

• Added new fields to the /contractors/ endpoint group:

  New `/contractors/` fields

• Implemented new API rate limits: 1M requests/month, or ~33k requests/day.

✨ New:

• Added first_seen_date to the permits table
  • this represents the date when the the permit itself was first retrieved for the Shovels platform

✨ New:

• Added Address Profiles
• Added Free Forever plan in place of a Free Trial.

🐞 Bugfix:

• Fixed issue where Geography Profiles (City, Jurisdiction, County, etc) filters for Property Type and Permit Category weren’t updating counts correctly.

✨ New:

• Added new /v2/{geography}/{geo_id}/metrics/monthly endpoint.
• Added new /v2/{geography}/{geo_id}/metrics/current endpoint.

💥 Breaking:

• Removed /v2/{geography}/{geo_id}/metrics endpoint.

• Changed monetary value data types from decimal integer (eg, dollars cents). This will affect the following parameters:

  /permits

  /contractors

  /contractors/{id}/metrics

  /{geography}/{geo_id}/metrics/{interval}

• Changed percentage data types from decimal integer (eg, 0.75 75). This will affect the following parameters:

  /permits

  /contractors

  /contractors/{id}/metrics

  /{geography}/{geo_id}/metrics/{interval}

🐞 Bugfix:

• Fixed issue where address object’s latitude and longitude coordinates were occasionally reversed, and built in anomaly detection and correction for future processing.
  • This fix continues downstream to API and Online records.

✨ New:

• Added employees to Contractor Profiles.
• Added links to Jurisdiction Profiles for all permits.
• Added links to City Profiles and County Profiles for all geo-related profiles.

🐞 Bugfixes:

• Fixed password reset bug.

✨ New:

• Added new /v2/contractors/{id}/employees endpoint.
• Added new /v2/addresses/{id}/metrics endpoint.

⚠️ End of Life:

• We will DEPRECATE the API V1 by the end of January 2025.

🐞 Bugfixes:

• Fixed metrics calculation algorithm on all endpoints.

💥 Breaking:

• contractors table:
  • first_seen_at renamed first_seen_date.
    • Data type changed DATE.
  • reviews renamed review_count.
• residents table:
  • Foreign key changed from permit_id address_id.
    • Please note: The address_id field is also the foreign key for the permits table, allowing easier joins between residents and permits tables.

✨ New:

• Added +2M permits in Texas.
• Added metric visualizations to Contractor Profiles.

🐞 Bugfixes:

• Corrected issue where Contractor Profiles wouldn’t load properly.

✨ New:

• residents endpoint.

🚀 Upgrades:

• Improved address validation for all geography-related fields, in line with US Census data.

💥 Breaking:

• Casing standardizations (please update accordingly if your logic is case-sensitive):
  • property_type values are now lower case, eg residential.

✨ New:

• first_seen_at column in the contractor table.
• employees table, linked to contractors table:
  • Includes a wide range of firmographic data for individual employees.
  • See “Schema Updates” below for the full list of columns.

🚀 Upgrades:

• additional columns in the residents table. See “Schema Updates” below for the full list.

💥 Breaking:

• Casing standardizations (in addition to the changes outlined in the API Section previously).
  • property_owner_type values are now snake_case, eg individual or company_owned.
  • owner_name, owner_street, owner_city values are now upper case, eg JANE DOE, MAIN ST, or AGAWAM.
  • applicant_name, applicant_street, applicant_city values are now upper case, just like owner_$ above.
• (PLANNED) Replacement of permits_ids with address_id in the residents table (please update your pipelines that use this foreign key value.)
  • This change will go into effect January 2025.