Big Data Platform Developer Knowledge / Skills :
Platform administration like Hadoop / Elastic Cluster, Cloudera Manager, Cloudera Director in Microsoft Azure cloud and on premise.
Knowledge on Kerberization & Sentry Implementation
Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).
Exposure in handling to huge quantities of data by taking advantage of both batch and speed methods.
Strong knowledge of and experience with statistics; potentially other advanced math as well and good working knowledge of SQL ability.
Experience in processing large amounts of structured / unstructured data.
Capability to design and document solutions independently, do Code reviews, Ensure standard Quality Assurance activities are completed on time.
Mandatory Skills :
Minimum 5+ years of working experience in the Apache Hadoop framework, HDFS, Map Reduce, Hive, Flume, Sqoop, Oozie, Spark, Scala, Spark Streaming, Kafka and HBase.
Minimum of 3+ years of experience in Big Data and batch / real-time ingestion and analytical solutions leveraging transformational technologies.
Good understanding about Lambda architecture
Implements security and recovery tools and techniques as required.
Ensures all automated processes preserve data by managing the alignment of data availability and integration processes.
Having exposure on Microsoft Azure Administration with Cloudera, Cloudera Director, Kerberization & Sentry
Having exposure on the Cloudera Manager, Cloudera Navigator, Hue & Shell Scripting
Having exposure on Microsoft Azure Cloud Migration from on premise to cloud
Kafka & Flume - Administration
Having exposure in automated deployment, monitoring and DevOps adoption.
Having exposure on ADLS, Network Segmentation, Azure & on premise network components, Azure Resources / cost optimization & Performance Optimization
Elastic Search & Elastic Search Cloud experience would be added advantage
Automated workflow migration would be added advantage
Integrating Hadoop with Oracle and NoSQL databases
CRITICAL COMPETENCIES (MAXIMUM OF 6)
Technology Doers / Full stack developers
Drive Efficiency and leaders in cutting edge technologies
Technical Capability development
Transparency and Collaboration
Self-driven and Self-learning
Problem solving and decision making
Appetite to develop in multiple platforms
ref : hirist.com)