Senior Data Engineer – Data Analytics & AI

Location: Pittsburgh

Job Type: Full Time / Permanent

The Senior Engineer is responsible for the design, development, implementation, and support of ETL processes for Data Analytics & AI. Works are part of a team to develop data pipelines, data transformation logic in ADF, python or Scala and other technologies to move data from a variety of operational platforms into Azure cloud. Conduct performance tuning of ETL processes for large and medium volumes of data, develop and oversee monitoring systems to ensure data loads complete on schedule and data is accurate. Assume responsibility for resolving production support incidents on data load processes and related analytic tools. Participate in project scope identification, design, development, testing and deployment-related activities in coordination with other members of the Data team, and across. Develop and improve standards and procedures to support quality development, testing, and production support. Participate in the on-call duty phone rotation. Provide leadership on projects and other Data initiatives as assigned. The Senior Engineer will be competent to complete most Data tasks independently and provide guidance to other members of the team as required. Position requires on-call and off-hours support as needed for operations and projects.

Responsibilities: • Designs and develops quality Data Warehouse solutions • Develops high quality, scalable data pipelines and data processes in Azure distributed cloud environment • Conducts testing, code reviews, data integrity, and performance optimization • Creates and maintains technical design documentation • Lead requirements gathering for data modelling and contribute to data architecture • Creates and promotes writing good quality code for accurate data • Supports developers, data analysts, business partners, and data scientists who needs to interact with data platform • Responsible for production support, including analyzing root cause and developing fixes to restore ETL and data operational readiness, planning and coordinating maintenance, conducting audits, and validating jobs and data • Mentors other team members, cross-trains and provides guidance • Solid understanding of work estimation process to lead large/complex estimation activities • Meets expectations in meeting deadline within budget, schedule, and appropriate quality • Adhere to enterprise architecture standards and contribute to making development and testing standards • Maintain pipelines in a git repository • Learn our Domain and contribute to solving data problems in an experienced manner • Contribute to a collaborative work environment within and across teams.

Basic Qualifications: • (These are the minimum qualifications you need to be considered for the job) If you have some but not all the requirements listed, please do not count us out. Studies show that some individuals avoid applying for jobs if they do not meet every qualification. We are committed to finding the right person for each role, not just checking off a list. We encourage you to apply as you might be the right fit for this role or other exciting opening. • Bachelor’s Degree in Computer Science or a related technical field • 7+ years of overall experience in Data Warehouse design and data modeling patterns (on-premises or cloud) • 7+ years of overall experience in developing SQL/Synapse data warehouses and T – SQL coding • 5+ years of experience in developing/supporting a data platform in Azure with data lake, Azure SQL server or Synapse • 5+ years of cloud experience with Azure/AWS and Databricks • 5+ years of experience with ETL tools such as ADF or equivalent • 5+ years of experience in python or Scala or any other language in distributed cloud environment. • Strong experience in performance tuning of ADF jobs, SQL, with medium and large volumes of data • Highly skilled in ETL tools such as SSIS, Informatica, Talend, AWS Glue, Azure Data factory or equivalent • Expert in creating T SQL or pgSQL or PL/SQL or equivalent in processing big data • Well-rounded in working in a DEV Ops environment supporting processes in data platform in supporting business units • Thorough knowledge in core data concepts providing solutions for business use cases in distributed computing environment • Expert level knowledge in writing python or Scala code in a distributed computing environment handling big data loads in lake house and delta lake environment • Strong knowledge in performance improvement methods in data processes • Experience working in an agile Data Warehouse team with 5+ members • Knowledge about BI tools such as power BI or equivalent in supporting Data Warehouse development, testing and operational support activities • Excellent written and verbal communication skills • Ability to work independently, handle multiple tasks simultaneously and adapt quickly to change with a variety of people and work styles • Must be capable of fully yet concisely articulating technical concepts to non-technical audiences • Keen on learning new concepts and keep up to date with emerging technical stack.