Big Data Engineer

Job Details

We are seeking a data engineer with skills to play a critical role in developing our Big Data Analytics Platform. if you are tech-passionate with self-confidence looking for a new opportunity, we offer an excellent position in the Digital Technology team within VF.

We are looking for a candidate with a very good understanding of Big Data Solution with current experience as a developer/engineer. Main responsibility is to work side by side with the other team member to develop and deliver efficient big data solution. Understanding of agile methodology is a mandatory skill.

The successful candidate will also partner closely with technical experts – including, but not limited to Architects – in defining reusable patterns, developing code standards/guidelines, and creating a review process that ensures quality across the development pipeline.

PRIMARY ACCOUNTABILITIES
* Evaluate and recommend key technology elements for the Analytics platform
* Collaborate with AWS Solution Architects to ensure the technical direction
* Development of big data pipeline
* Work in a team with other data engineer to release deliverable for the business
* Participate in an Agile squad (scrum team),

RELATIONSHIPS

Internal:
Scrum Masters, Product Owners, Lead Engineer, Solution Architects, Service team, DevOps Engineers, Operations Personnel, Business Owners

External:
Suppliers, Consultants

QUALIFICATIONS

Experience
* 3+ years overall software development in Big Data
* Previous experience as a Software Engineer/Big Data Engineer
* Familiarity with JIRA & Confluence or similar tracking and management tools
* Excellent organizational, verbal and written communication and the ability to present information in a clear, concise and complete manner
* Self-starter, creative, enthusiastic, innovative and collaborative attitude
* Ability to prioritization task based on a sense of urgency and accuracy

Skills
* Development and support of performance pipelines
* Consolidation of different sources of data (API, SQL Database, CSV, S3 and FTP files etc) into a centralized data store.
* Agile development of Data Lake / Data Warehouse and ETL / EMR (Glue, Hadoop, Spark, YARN, HUE, REST APIs, HIVE).
* Redshift and BigQuery as data warehouse system.
* Experience with RDBMS (MySQL, Oracle or DB2..) and SQL/DDL Language, and NoSQL (DynamoDB)
* Development of message queue driven systems (Amazon SQS, SNS and Lambda based functions)
* Development of streaming system (ie: Kinesis Stream, Kinesis Firehose)
* Python development, PySpark development
* Lambda development in Python, Node
* Ability to perform code review and technical design review.
* AWS Knowledge is mandatory

Tagged as: