Job Description

An FRE-listed company that has single-handedly transformed the automation and transportation industry with their world-class range of products, is looking for a Senior Data Engineer. The engineer will be helping other software engineering teams and the data scientist in the data engineering team with feature engineering, data wrangling, and data purification. The company is creating electronic cars and automotive technology to help consumers accelerate the transition to sustainable mobility. The company has managed to secure more than $13.2bn in funding so far.

Job Responsibilities:

• Among other things, use Python, SQL, AWS Glue, lambdas, API Gateway, Kafka, data build tool (dbt), and Apache Spark on EMR to create a modular collection of data services
• Utilizing frameworks such as PySpark, create automated pipelines for unit and integration testing
• Using Gitlab CI and AWS Code Pipeline/CodeDeploy, create and manage CI/CD pipelines
• Schedule and automate tasks with Managed Airflow
• Create the ODS and reporting schemas, then import the data into Snowflake or AWS Redshift
• Using Apache Deequ and data observability technologies such as Splunk, DataDog, CloudWatch, and Montecarlo, design and implement data quality management services
• Offer a range of query services using server-sent events, Athena/Presto, and REST
• Enable numerous technical, operational, and business teams at the client to share data in a fast, efficient, and secure manner in order to create world-class data products
• Utilizing tools such as Collibra/Alation, configure and set up enterprise data lineage, metadata management, and data catalog support
• Make sure deployment builds are green, and collaborate with senior leads and program management to exhaust scheduled deliverables within a sprint cycle