Senior Data Engineer
- Hiring from
- Colombia
- Work type
- Remote
- Posted
625,541 remote jobs, straight from company career pages
100% free · New jobs every minute
Show job descriptionHide job description
WHAT WE DO
Founded in 2007, Growth Acceleration Partners (GAP) is a consulting and technology services company. We consult, design, build and modernize revenue-generating software and data engineering solutions for clients. With modernization services and AI tools, we help businesses achieve a competitive advantage through technology. GAP’s remote, integrated engineering teams use end-to-end solutions to innovate and align with your business goals. We have 600+ English-speaking engineers based in Latin America and approximately 20 U.S.-based engineers. With some of the highest customer satisfaction scores in the industry, GAP’s focus is customer and employee success.
GAP is a woman-owned company headquartered in Austin Texas. We are a values-based company focused on growing our people by investing in education, onsite English classes and training in the latest technologies, including AI, data analytics and machine learning. Our goal is to provide solutions for our customers that help them achieve critical business outcomes, while enabling our GAPSters and our communities to attain long-term success.
Summary
We are looking for a Senior Data Engineer to build and operate the data pipelines that consolidate portfolio company data into a single, business-aligned model on Databricks for a company in the private equity and asset management industry. You will own pipelines end to end, from incremental ingestion through Bronze, Silver and Gold layers to serving data for BI reporting and internal applications. This role also supports a key migration from a legacy system, including historical loads and financial reconciliation before cutover. You will work with a high level of autonomy and partner closely with finance, reporting and application teams.
Education
Bachelor’s Degree in Computer Science, Software Engineering, Information Systems, or a related technical field.
Professional Experience
5+ years of experience building production data pipelines, including 2+ years on Databricks with Delta Lake
Hands-on experience implementing medallion architectures (Bronze, Silver, Gold) with incremental processing
Proven track record in data quality and reconciliation, including tracing incorrect figures back to their source
Experience supporting data migrations from legacy SQL Server environments to modern lakehouse platforms
Key ResponsibilitiesBuild incremental ingestion pipelines using watermarks and change timestamps, safely handling late-arriving data and retaining full raw history
Model and transform data across Silver and Gold layers, including deduplication, unit and currency conversion, and quarantine of suspect records
Implement dimensional models with fact tables, slowly changing dimensions and reference data optimized for Power BI and application use
Support the migration from the legacy system, loading and freezing historical data and reconciling figures before cutover
Publish curated data to downstream consumers through Delta tables and Postgres-based synced tables for applications
Orchestrate, monitor and document pipelines, manage access and lineage in Unity Catalog, and maintain runbooks
Required Technical Skills
Strong experience in several of the following areas:Lakehouse: Databricks, Delta Lake, MERGE, change data feed, medallion architecture
Languages: SQL (Spark SQL and T-SQL), Python, PySpark
Data Modeling: dimensional modeling, fact grain definition, surrogate keys, slowly changing dimensions
Governance & Quality: Unity Catalog or a similar governance layer, data quality checks, reconciliation, lineage
Performance: BI workload tuning, file layout, clustering, query plan analysis
Nice to HaveExperience with private equity, asset management or fund administration data, such as portfolio monitoring, valuations and cash flows
Knowledge of financial reporting concepts such as LTM and YTD periods, fiscal year ends, restatements and FX conversion
Experience with Lakeflow (Delta Live Tables), Databricks Workflows, or Databricks Lakebase and PostgreSQL
CI/CD and infrastructure as code for data pipelines, such as Databricks Asset Bundles, on Microsoft Azure
Soft SkillsAdvanced English proficiency
Clear written communication, able to explain data issues and trade-offs to non-technical users
Strong attention to detail and ownership of data accuracy
Collaborative mindset when working with vendors, finance and reporting teams
At Growth Acceleration Partners, we're an equal opportunity employer committed to building a diverse and inclusive team. We value everyone's unique background, and we provide equal opportunities regardless of race, color, creed, religion, sexual orientation, gender identity, age, national origin, disability, marital status, veteran status or any other personal right protected by law. We foster a culture of belonging and strive to provide a welcoming environment where everyone feels safe to contribute and grow.
