The Lead Data Engineer is responsible for aggregating data from various systems into a central data lake and warehouse to be used for statistical analysis, reporting, and dashboards.
This position will utilize modern cloud data warehouse technology and works with both unstructured and structured data. This is not a traditional data warehouse position and will not involve traditional schema-based relational database design.
The successful candidate will have an understanding... of cloud data warehouse technology, as well as how to organize and store data for use with advanced analytics.
Roles and Responsibilities
Data engineer with strong experience in Data Profiling, data cleansing.
Should have good understanding of Data Governance, Collibra Operating Model, Concepts, Collibra building catalogs, Designing etc.
De-duplication and data quality assessment.
Data governance & data quality management roles with hands on experience in transforming data delivery architecture.
Experience with Data Management. Injection, storage, wrangling and rendering.
Experience in Mulesoft or Dell Boomi Integration related projects
Experience in Java or Groovy Scripting
Good Understanding in BPMN workflows .
Experience with ETL or ELT with Any of the Cloud Platform like AWS (EMR, RDS, Redshift, DynamoDB, S3 etc), Azure(Function Apps,Transformation ..) , GCP
Experience with Agile / Scrum is valuable.
Experience more than one data warehousing platforms
Experience with object-oriented / object function scripting languages : Python, Java, Scala, etc., a plus.
GitHub / Bit Bucket will also preferred.
Desired Candidate Profile
Bachelor’s Degree plus at least 8-10 years of experience managing data warehouse and / or business intelligence systems.
An advanced degree or certifications in a related field is a plus.
3+ years of experience with data engineering, ETL, data warehouse development, or data lake development.
Advanced experience implementing complex Data Architectural designs with complex data orchestration and integration needs while maintaining trust and quality of information delivered.
Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databasesRead more