Data Engineer
Job Description
- Data Pipeline design and implementation
- Design and implement batch or real time data pipeline
- update data warehouse
- build machine learning models
- automate data analytics workflow
- DevOps
- Automate deployment services on data platform
- Tech support for Data Analysts and Data Scientist
- Help Data Analysts proceed data analytics projects
- Help Business Analyst and Data Analyst create BI dashboard for their KPI
- Design and create required data warehouse resources (tables/views)
Job Requirement
Must-have technical skills
- Python or similar scripting language
- Data warehouse - Cloud services (Snowflake/BigQuery/Redshift) or OSS (Presto/Hive)
- SQL
- Workflow automation tool (Airflow, Luigi, etc)
Nice-to-have skills
- Big Data processing: Cloud Services (AWS EMR, AWS Glue) or OSS (Spark, Beam, etc.)
- Data science and machine learning using Python
- CI/CD pipeline: AWS Code Build, AWS Code Pipeline, Github Actions, Circle CI.
- Infra as Code: AWS CDK, Terraform, etc.
- BI (Tableau, Looker, Superset, etc.)