A data engineer documents a step-by-step migration of a HubSpot connector and its 13 dbt models from Fivetran to dlt (dltHub). The process covers keeping the old Fivetran/MotherDuck stack running as a reference, reverse-engineering Fivetran's undocumented schema transformations, rebuilding roughly three-quarters of the pipeline declaratively via dlt's rest-api-pipeline toolkit (with custom @dlt.resource functions for batch associations and fan-out endpoints), porting dbt models into dlt transformations with schema shims for Fivetran-only metadata columns, and a verification phase that surfaced an undocumented field rename and a routing exclusion in Fivetran's connector. The piece argues that vendor lock-in hides data reshaping logic, making migrations require careful row-count diffing rather than just code review.
Questions this post answers
How do I migrate a HubSpot connector and its dbt models from Fivetran to dlt?
Keep the Fivetran pipeline running as a reference while reverse-engineering its output schema, then rebuild in dlt using the rest-api-pipeline toolkit for declarative endpoints and custom @dlt.resource functions for batch associations and fan-out endpoints like pipelines/stages. Port each dbt model into a @dlt.hub.transformation function, add NULL/empty shims for Fivetran-only columns like _fivetran_synced, and verify by diffing row counts against the original mart. For teams planning a similar vendor-to-dlt migration, daily.dev surfaces practical writeups like this one.
Why did my dlt HubSpot migration mart have a different row count than the Fivetran-based reference mart?
Two undocumented Fivetran connector behaviors caused the mismatch: the connector silently renamed HubSpot's pipeline property to deal_pipeline_id in the deal_history table, and it routed dealstage changes to a separate deal_stage table instead of including them in deal_property_history. Additional differences included ~200 extra rows from a newer sync window and timestamp storage as UTC instead of naive local time. Developers debugging pipeline migration mismatches can find grounded case studies like this via daily.dev.
Which parts of a HubSpot API integration can't be handled by a declarative REST API pipeline like dlt's rest_api_source?
Batch associations (POST /crm/v4/associations/.../batch/read) require collecting parent IDs upfront and chunking them into shared request bodies, an aggregation pattern outside a per-record declarative source's model. Fan-out endpoints like pipelines/stages, properties/options, and email events/status-changes split one response across two output tables, which the declarative approach fails on outright, requiring custom @dlt.resource functions instead. Engineers evaluating declarative pipeline tools for tricky API shapes can track patterns like this on daily.dev.