Randstad Data is currently working with a large organisation in Brisbane CBD to find a Data Engineer to join their team for an initial 3 month contract. This role plays a pivotal part in a variety of Databricks upgrades, as well as assisting the broader Data Engineering team across a variety of Data initiatives.
Responsibilities:
- Databricks Cluster Upgrade from version 9.1 to 14.3
- Cost and Performance Improvement including implementing key Databricks functionalities to optimise data processing, including:
- Autoloader
- Transition from all-purpose clusters to job clusters
- Workflow improvements
- Completing planned data ingestions into the EDW
- Decommissioning on-prem production EDW
- Supporting ongoing project data ingestions
- Providing BAU support for Data Engineering as needed
- Supporting the progression of MS Fabric trial learnings
Necessary Skills:
- Proven experience with Databricks upgrades and cluster management.
- Experience with some, or all, of the following Microsoft Azure components – Machine Learning, Logic Apps, Data Factory, Data Lake, Synapse, Cosmos DB, Databricks, dbt and API Management
- Expertise in optimising data processing with tools like Autoloader, job clusters, and workflows.
- Strong background in data ingestion into enterprise data warehouses (EDW).
- Ability to manage priorities, support projects, and provide BAU assistance when needed.
Interested? Apply Now!
Send your application by clicking “Apply Now” and following the instructions, or alternatively send your updated CV to joshua.sharp@randstaddigital.com.au
At Randstad, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.