Databricks Engineer

About Us Exavalu is a US based consulting firm specialized in Digital Transformation Advisory and Digital Solution Delivery for select industries like Insurance, Healthcare and Financial Services. Our Headquarters is in California, US with multiple offices across US, Canada, and delivery centres in India. We were founded by Industry executives and Consulting principals with deep industry experience that allows us to bring advisory strength and solution expertise to clients. We’re in a hyper growth trajectory growing at over 100% year on year. Join our diverse and inclusive team where you will feel valued and motivated to contribute with your unique skills and experience. Exavalu offers permanent remote working model as we believe in going where the right talent is. Job Description This is a remote position. Data Pipeline Development: Design, build, and optimize robust ETL/ELT pipelines using Azure Databricks, Spark, and SQL. Data Processing & Transformation: Utilize Python, PySpark, and SQL to clean, transform, and aggregate complex data for analytics. Azure Data Lake Management: Manage and optimize data storage and retrieval in Azure Data Lake Storage (ADLS) Gen2. Delta Lake Implementation: Implement Delta Lake for ACID compliance, data versioning, and high-performance data lake operations. Integration with Azure Services: Integrate Databricks with Azure Data Factory for orchestration, Azure Synapse Analytics for warehousing, and Azure Key Vault for security. Performance Optimization: Monitor, troubleshoot, and optimize Databricks clusters and spark jobs to manage costs and performance. Data Security & Governance: Implement Role-Based Access Control (RBAC), data encryption, and data lineage tracking. Requirements Strong data engineering skills on Azure, ADF, Synapse and Databricks Strong experience of Python/Pysprak, SQL Benefits Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely. We also offer a welcome back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons. At Exavalu, we are committed to building a diverse and inclusive workforce. We welcome applications for employment from all qualified candidates, regardless of race, colour, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity or any other status protected by applicable law. We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career