Data Engineer
Karya, Bengaluru +1
The role
This role runs live client projects on the Karya platform. The work is the execution layer above the core product: project workflows, data jobs, checks, reports, and delivery. You will work with Operations and client teams on questions of progress, quality, pay, and files, and with platform engineers when a fix belongs in the product. Worker payouts and client deliverables depend on getting these checks right.
What you will do
- Run and monitor data jobs on remote Linux servers, and recover failed jobs from logs and process state.
- Configure project workflows, including the handoff from automated processing to human tasks, and track what
is in progress, held, or ready to deliver.
- Write PostgreSQL and ClickHouse queries for throughput, completion, quality, pay, and delivery status.
- Apply completion and quality rules to access, pay, and delivery decisions, after a sample check.
- Write Python, SQL, or Bash scripts for those checks and decisions, and keep them in the repository.
- Package datasets to client specifications, move them through cloud object storage, and record what has shipped.