About the Team/Role
We are seeking an experienced Data Engineer to play a critical role in the development of WEX's data & analytics capabilities. You will be part of an organization focusing on the development and delivery of data solutions and capabilities for WEX’s data platform. You’ll be part of a team that is responsible for:
- Creating optimized data pipelines using Qlik Replicate, Fivetran, custom code using Python, and Snowpipes
- Working with stakeholders to understand business requirements and then implement SQL-first transformation workflows to deploy analytics code using dbt (data build tool)
- Designing efficient data marts that are catered towards the needs of very specific business units, functions, or departments
You’ll be part of a collaborative scrum team that consists of an Agility Engineer, Technical Program Manager, Data Engineers, QA, and DevOps. You’ll also be supported by a manager that is here to listen and help you grow.
The successful candidate is motivated by thinking big data, is technically proficient, and enjoys working in a fast-paced environment. You care deeply about the veracity (i.e. consistency, accuracy, quality, and trustworthiness) of data. You enjoy designing, maintaining, and optimizing data pipelines & infrastructure, for data collection, management, transformation, and access. You also enjoy understanding complex business requirements and translating them into data models for the end users.
How you’ll make an impact
- Are a strong critical thinker with analytical and problem-solving abilities
- Are self motivated and able to work independently with minimum supervision
- Bring thought leadership to your area of responsibility and enjoy staying ahead in your field
Our core stack for this position consists of dbt, Snowflake, Python, Airflow, Docker, Qlik Replicate, Fivetran, and several AWS services. You possess the following skills and experiences:
- Solid understanding of dbt to perform data transformation tasks (3+ years)
- Strong understanding of data design principles and dimensional data modeling (2+ years)
- Advanced SQL skills (3+ years) and understanding of query optimization strategies in Snowflake
- Fundamental understanding of DAGs and operators in Airflow, with 1+ years of hands-on experience with action and sensor operators
- Fundamental understanding of Docker and deploying code as a container
- Solid understanding of basic programming concepts in Python or similar modern languages (2+ years)
- Solid understanding of Qlik Replicate and Enterprise Manager with 1+ years of experience managing endpoints and streaming CDC (change data capture) tasks
- Solid understanding of Fivetran with 1+ years of experience ingesting files and SaaS (e.g. Salesforce, Workday) data sources
- Fundamental understanding of the following AWS services:
- Elastic Compute Cloud (EC2) - performance monitoring & optimization and knowledge of instance types
- Virtual Private Cloud (VPC) - provisioning of virtual networks and all its basic components (security group, network access control list, route tables, subnet, etc.)
- Elastic Container Registry (ECR) - deploying of application images and artifacts
- Elastic Map Reduce (EMR) - managing and running PySpark code in a distributed processing framework; including troubleshooting and cluster optimization
- S3, IAM, Secrets Manager, AWS CLI
Note: Qlik Replicate and Fivetran experiences can be substituted with ETL/ELT data pipeline development experience using Python (or similar tools).
Bonus: You have successfully implemented CI/CD using Jenkins or Github Actions in a production environment.
Experience you’ll bring
- At least 5 years of experience as a Data Engineer creating data pipelines using tools and custom code
- At least 3 years of experience with dbt (or similar tool) developing complex data models using macros and Jinja
- BS in a technical or quantitative field; or you can make us feel intensely confident that you don’t need one