Overview:
Randstad Digital are partnering with a firm in the search for a Technical Lead (Lead Data Engineer) to join their team on a 2 year fixed term basis. Our client is building a new modern data and analytics platform using the Azure Cloud stacks (Azure Data Factory, Azure Databrick and ADLS).
In this transformation, you will play a critical role as the Lead SME in the design, development and implementation of data solutions on Azure Cloud and Azure Databricks in a greenfield environment.
What we are seeking:
We are seeking an experienced candidate with a proven track record in delivering robust, scalable and high quality data solutions on Azure Cloud and Databricks in large, enterprise environments.
With a passion for driving best practices and efficient data systems, you will have technical leadership skills across mentoring a skilled team, driving continuous improvement and aligning a data roadmap into effective project delivery.
Responsbilities:
Design and Development:
... - Lead the design and implementation of data pipelines, ETL processes and data models on Azure Databricks.
- Ensure the adoption of best practices in data engineering, including data quality, optimisation and security
- Translate high-level architecture into actionable implementation plans
Implementation & Best Practices:
- Ensuring solutions are scalable, reliable and secure along with mentoring team members
- Conduct code reviews, design reviews and ensure the team follows best practices in version control, CI/CD and DevOps for data engineering
Collaboration & Communication:
- Collaborate with cross-functional teams, including data architects, data modellers, analysts, and IT operations, to ensure seamless integration of data solutions.
- Communicate technical concepts to non-technical stakeholders, providing insights and recommendations to drive data-driven decision-making.
Governance Quality:
- Ensure data governance, security and quality standards are maintained throughout the data lifecycle
- Monitor and support data management initiatives, including within specific environments like Microsoft Azure/Databricks
Key Knowledge and Experience
- 6+ years of experience in data engineering, with a strong focus on Azure Cloud and Azure Databricks.
- Strong understanding of data architecture, data warehousing, data modelling, ETL processes and cloud based technologies.
- Demonstrated experience in mentoring technical teams
- Proficiency across Python, SQL and Scala
- Excellent problem-solving skills and the ability to work under pressure in a fast-paced environment
- Strong communication skills, with the ability to interact effectively with technical and non-technical stakeholders
Our client is looking to commence interviews ASAP with the successful candidate to be onboarded this year. For more information and a confidential chat, please reach out immediately to matt.andrews@randstaddigital.com.au with your updated cv.
At Randstad Digital, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.
