Senior Data Engineer responsibilities will include :
Data Acquisition : Design, build, manage and customise ETL processes / pipelines that both land data onto the platform and also provision data out, for consumption by APIs
Data Preparation : Bringing together & unifying unintegrated information from different sources
Data Exploration, Information delivery, insight generation, opportunity discovery, etc
Data Mining : Proactively analyse data and research areas of business interest to identify opportunities, propose data sets, analysis, data science products which could allow the company to gain a competitive advantage
Requirements and Qualifications
3-6 years experience in data / technology projects
A degree in computer science, engineering , or a related field.
Critical thinking, analytical mind, problem-solving capabilities, and high level of initiative
Strong interpersonal skills with ability to work both independently and as part of a team.
Technical skills :
Knowledge of logical and physical data modelling. Storage, sharding, partitioning
Knowledge of Python, and / or other languages such as R, Java, Scala, Spark
Advanced knowledge of SQL, and knowledge of NoSQL paradigms.
Knowledge of DevOps e.g. containerisation, logging, provisioning, CI / CD
Knowledge of AWS data tools like EMR, Lambda, Glue, Redshift and solutions
General business skills
Experience in interacting with stakeholders, gathering & refining requirements
Excellent written and verbal communication, and presentation skills.
Ability to complete projects and achieve results in an ambiguous work environment.