Databricks Specialist
We are looking for an experienced Databricks Specialist to build, run and maintain data pipelines
and curated data products on an enterprise Azure Data Lake that supports business-critical
reporting. The role covers the full path from raw data to trusted, reusable Delta tables: developing
Databricks jobs, choosing the right load method, managing downstream impact on dependent use
cases, and resolving failures in a live production environment. The specialist will work closely with
business data product owners, use case teams and platform engineers, under ticket-based change
control and disciplined documentation.
Department:
Managed Service
Employment Type:
Full-time
Experience:
1-3 Years
Workplace Type:
On-site
Number of Vacancies:
2
Location:
Dhaka
Deadline:
31 Oct 2026
Key Responsibilities
1.1 Data Product Development
- Design, build and maintain Databricks jobs (PySpark and SQL) that turn raw landed data into curated, trusted Delta tables.
- Select and implement the right load method for each data product: full refresh, SCD Type 1 (overwrite) or SCD Type 2 (historized).
- Build new data products end to end when new sources are approved: ingestion hand-off, processing, scheduling, alerting and documentation.
- Extend existing data products with new columns, tables or source data, and remove data that is no longer required.
1.2 Pipeline Operations and Reliability
- Monitor scheduled jobs, investigate failures and anomalies, and trace issues back through the data layers to their root cause.
- Reprocess historical data from the raw layer when a fix must apply to past
- Work with Azure Data Factory pipelines and Azure Data Lake Storage Gen2 to keep ingestion and processing aligned.
- Support incident recovery and root-cause analysis, and document permanent fixes for recurring issues.
1.3 Downstream Impact and Use Case Support
- Maintain the dependency map between data products and the use cases that consume them, and check every change against it before release.
- Make data available to use case teams, answer technical questions on data content and load logic, and support them as they build.
- Read and explain existing transformation code to business stakeholders, and apply logic changes only on the business owner’s instruction.
1.4 Performance and Cost Optimization
- Tune job schedules, run times, cluster configuration and compute cost to meet business
- Maintain data product workspaces, clusters, repositories and
- Recommend optimization opportunities (e.g. cluster rightsizing, incremental processing) with expected benefit.
1.5 Governance, Documentation and Collaboration
- Work through ticket-based change control on Azure DevOps, recording requirements, decisions and outcomes on each ticket.
- Document every data product, change and fix so that the platform remains traceable and
- Collaborate with data engineers, BI analysts, platform administrators and business stakeholders in sprint-based delivery.
Required Skills & Experience
1.1 Technical Skills
- 5+ years of data engineering experience, including hands-on production work in Azure
- Strong Apache Spark (PySpark) and SQL skills, with experience building and optimizing Delta Lake tables.
- Practical experience with load patterns including full refresh, SCD Type 1 and SCD Type
- Working knowledge of Azure Data Factory, Azure Data Lake Storage Gen2 and the Azure data platform.
- Experience with Databricks workflows, clusters, repositories and job scheduling, including performance and cost tuning.
- Familiarity with Git-based source control and CI/CD (Azure DevOps or GitHub).
- Strong debugging skills across multi-layer data platforms (raw, curated and consumption layers).
1.2 Professional Competencies
- Build-and-run mindset: comfortable owning both new development and day-to-day production stability.
- Careful, impact-aware approach to change in a shared platform where one data product feeds many consumers.
- Clear communicator who can explain data logic and technical issues to business
- Structured working style with disciplined documentation and change-control
- Able to work effectively with remote, international stakeholders and multiple dependent.
1.3 Preferred Qualifications
- Experience operating an enterprise Data Lake in the automotive, manufacturing or logistics
- Familiarity with Python-based data quality checks, Airflow or
- Exposure to Power BI or other BI tools consuming Databricks data
- Microsoft or Databricks certifications (e.g. DP-203, DP-600 or Databricks Data Engineer Associate/Professional).
Interview Steps
- Technical/Functional Interview
- HR Interview
- Offer