As a Data Engineer, you will be a key player in our data engineering team, responsible for designing, building, and maintaining our data pipelines and infrastructure. You will work on transforming raw data into structured formats to support advanced analytics and reporting. Your work will ensure the smooth ingestion, transformation, and organization of data in our data lakehouse so that it’s ready for data analysts to consume and transform into actionable dashboards and reports.
Responsibilities
- Design, develop, and optimize data pipelines to ingest data into our data lakehouse.
- Implement and manage ETL processes using Google Cloud Composer, Dataproc, and dbt.
- Develop data transformations within a medallion architecture (bronze, silver, gold layers).
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
- Implement data quality checks and monitoring.
- Monitor and optimize the performance of BigQuery datasets and queries.
- Document processes, designs, and architectural decisions.
- Design and develop a read-optimized data warehouse layer.
- Troubleshoot daily reporting pipelines
- Depending on the team, participation in an on-call rotation with additional compensation might be required.
Requirements
- 3+ years of experience in data engineering or related roles.
- Strong analytical skills with the ability to manipulate and interpret complex datasets.
- Proficiency in SQL and experience with databases such as MySQL and BigQuery.
- Experience in both Python and Scala is preferred
- Experience with data platforms like Airflow, Dataproc, Great Expectations.
- Excellent communication and interpersonal skills, with the ability to effectively collaborate with cross-functional teams.
Nice to haves
While not specifically required, tell us if you have any of the following.
- Conversational Japanese
- Understanding of cultural differences
- Knowledge and experience in chatbots, AI, marketing, and SaaS products
