Senior Data Warehouse Engineer
Location: West Pittsburgh, PA
Job Type: Full Time / Permanent
The Senior Data Warehouse Engineer is a key member of the Enterprise Data Warehouse team that leads the design and development of data extraction, transformation and loading (ETL) from disparate source systems. Technologies include (Kafka,MQ,Elastic) and utilize Java(Spring Boot/Spring batch), APIs etc to efficiently load data into Enterprise Data Platforms (Oracle/Google Cloud Platform) for reporting and analytics needs.
Education & Experience:
- Bachelor’s Degree in Computer Science, Software Engineering, Information Systems or Information Technology or related field required, or equivalent experience
- Create and maintain optimal data pipeline architecture for Data extraction and transformation based on source system configurations and Data Warehouse staging requirements.
- Identify, design and implement internal process improvements (automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc).
- Build the capabilities required for optimal extraction, transformation and loading of data from a variety of data sources using Java (Spring Boot / Spring batch) SQL, Python, Google Big Query and other Big Data technologies.
- Build re-usable and maintainable code with proper error handling. Act as a mentor to other members of the team.
- Preferred Experience:
- 3-5 years of experience
- Experience with Java (Spring Boot/Spring batch) , SQL, Python, C# ,Shell Scripting (KSH,Bash)
- Experience with Streaming Technologies (Kafka)
- Experience in extraction and loading of JSON,XML CSV file formats.
- Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores
- Experience with relational database design methodologies.