Data Engineer at Versent

Data and Insights, Full Time, Brisbane, AU brisbane engineering full-time
Description
Posted 1 months ago

We are the Australian AWS Consulting partner for the Year for the 3rd time and recently became a Snowflake and Talend Partner.
We believe in bringing craftsmanship back to technology, because as technology artisans, we passionately care about the quality outcomes we leave behind.

We have some of the best techs in Australia and due to client demand and we're on the hunt for experienced Data Engineers to come on board.

As our Data Engineer you will help deliver scalable, automated, repeatable ‘big data’ and analytic patterns within the AWS stack. Versent Data Engineers understand both data pipeline development, operations and management as well as delivery of automated data solutions that are ready for production operations.

Responsibilities:

  • From a development or operations background (or an existing DataOps Team)
  • Understanding what getting something “done done” looks like (documentation, testing, operational requirements, continuous improvement of process)
  • Experience with big data tools: Hadoop, Spark, Kafka, Kinesis etc.
  • Knowledge of relational SQL and NoSQL databases, including Snowflake, PostgreSQL, Cassandra, etc.
  • Extensive experience with data pipelines and workflow management tools: Azkaban, Luigi, NiFi, Airflow, etc.
  • Experience with AWS cloud data services: EMR, RDS, Redshift, Kinesis, Glue
  • Previous exposure to stream-processing systems: Storm, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: SQL, Python, PySpark, Scala, etc
  • #LI-SC1