[Remote] Databricks Engineer :: Washington, DC (Remote)
Note: The job is a remote job and is open to candidates in USA. ARK Solutions, Inc. is seeking a Databricks Engineer to design, develop, maintain, and optimize scalable data pipelines and ETL processes. This role is crucial for modernizing data platforms and supporting enterprise data initiatives.
Responsibilities
- Create and maintain ETL pipelines and/or migrate existing legacy pipelines (e.g., synapse) to Databricks
- Develop scalable data processing solutions using Python, PySpark, and SQL
- Perform data transformation, cleansing, and validation to ensure data quality and consistency
- Optimize data workflows and Spark jobs for performance and efficiency
- Collaborate with business stakeholders, data architects, and development teams to understand requirements and deliver effective solutions
- Monitor, troubleshoot, and resolve data pipeline issues
Skills
- Hands-on experience with Databricks
- Strong programming skills in Python and PySpark
- Strong SQL development and query optimization experience
- Experience designing, developing, and maintaining ETL pipelines
- Experience working with large-scale data processing and analytics solutions
- Strong analytical, troubleshooting, and problem-solving skills
- Excellent communication and collaboration abilities
- Databricks
- Python
- PySpark
- SQL
- Bachelor's Degree
Company Overview
Company H1B Sponsorship