Data Engineer (Elasticsearch + Datawarehousing)

Particle41 is seeking a talented and versatile Data Engineer to join our innovative team. As a Data Engineer, you will play a key role in designing, building, and maintaining robust data pipelines and infrastructure to support our clients' data needs. You will work on end-to-end data solutions, collaborating with cross-functional teams to ensure high-quality, scalable, and efficient data delivery. This is an exciting opportunity to contribute to impactful projects, solve complex data challenges, and grow your skills in a supportive and dynamic environment.

Requirements Gathering and Analysis

• Collaborate with product managers, and other stakeholders to gather requirements and translate them into technical solutions.
• Participate in requirement analysis sessions to understand business needs and user requirements.
• Provide technical insights and recommendations during the requirements-gathering process.

Skills and Experience We Value

• Bachelor's degree in Computer Science, Engineering, or related field.
• Proven experience as a Data Engineer, with a minimum of 3 years of experience.
• Proficiency in Elasticsearch and Python programming language is a must.
• Experience with database technologies such as SQL (e.g., MySQL, PostgreSQL) and NoSQL (e.g., MongoDB) databases.
• Strong understanding of Programming Libraries/Frameworks and technologies such as Flask, API frameworks, Data warehousing/lakehouse, Principles, Database, ORM, Data analysis, Databricks, Pandas, Spark, PySpark, Machine learning, OpenCV, Scikit-learn.
• Utilize Java to build and enhance backend systems, particularly for integration with Elasticsearch and databases.
• Develop APIs, microservices, and automation scripts as needed.
• Utilities & Tools: logging, requests, subprocess, regex, pytest
• ELK stack, Redis, distributed task queues
• Strong understanding of data warehousing/lakehousing principles and concurrent/parallel processing concepts.
• Familiarity with at least one cloud data engineering stack (Azure, AWS, or GCP) and the ability to quickly learn and adapt to new ETL/ELT tools across various cloud providers.
• Familiarity with version control systems like Git and collaborative development workflows.
• Competence in working on Linux OS and creating shell scripts.
• Solid understanding of software engineering principles, design patterns, and best practices.
• Excellent problem-solving and analytical skills, with a keen attention to detail.
• Effective communication skills, both written and verbal, and the ability to collaborate in a team environment.