- Looking for an intermediate Data engineer for a client in the early stages of their start-up.
- Role focus: Build and maintain data pipelines for batch and streaming data, integrating machine learning and large language models, Developing data processing pipelines using Python and PySpark, using Delta Lake for scalable data management, integrating machine learning and language models into data pipelines, and optimizing data ingestion from various sources.
- Required Skills: Bachelor’s/Master’s in relevant fields; proficiency in Python, PySpark, Delta Lake, and Databricks; strong data modelling and ETL skills; excellent communication.
- Additional Skills: Experience with machine learning, MLOps, Azure tools, CI/CD practices, and real-time data processing (e.g., Apache Kafka).