Data Engineer
Viaduct, Slovenia +1
Viaduct has this job in 1 other place
The role
As an early data engineer at Viaduct, an analytics and ML platform company, your work is critical to our success. You are responsible for ensuring data quality and reliability in every part of our ETL pipeline, from ingestion to client integrations.
What you will do
Responsibilities
- Creating and supporting batch, incremental, and real-time ETL pipelines
- Standardizing ingestion, validation, and cleaning processes across clients
- Automating data validation to increase data quality
- Managing and evolving schemas in all parts of our pipeline
- Monitoring, tuning, and optimizing Spark jobs
- Becoming a domain-expert in connected vehicle data
Security and Privacy Responsibilities
- Follow our policy and procedure documents related to security and privacy
- Follow the security and privacy guidelines in the Employee Handbook