Lead BigData Software Engineer
Luxoft Poland
Antwerp, Belgium
3 dagen geleden
source : Just Join IT

Apache Spark (regular)

ELK (advanced)

Hortonworks HDP / HDF (advanced)

big data technologies (advanced)

Project Description

We are GBM Surveillance IT. We work within investment banking IT organization and with other global businesses to protect our stakeholders from illegal or prohibited activity across channels, including but not limited to trading activity, audio communications and electronic communications.

We work in small teams, bringing development, testing, business analysis, QA and system reliability together. We have started on our journey towards working in a truly agile, no-silos environment.

Our teams write their code, write automated test harnesses for their code, use automated mechanisms to deploy their code, and are accountable for their code in Production, including monitoring to ensure they work uninterrupted while delivering new code.

All the while they also work closely with business analysts and business stakeholders to deliver value in a steady, incremental stream.

For us, DevOps is not just about technology automation (automated build, test and release). It is about a cultural focus on working cross-functionally and delivering in small increments.

We do need people who are 100% focused on automated delivery and are capable of working in dynamic, cross-functional teams to make this happen!

As we're at the start of our Agile / DevOps journey, joining us will enable you to make a lasting impression in a key regulatory / compliance function in one of the world's largest banks.


  • Work with the Product Owner to understand stakeholder requirements
  • Responsible for building and managing data processing components and storage technologies
  • Responsible for onboarding new data sources and constructing data processing pipelines to present required views of data for application developers
  • Work with Data Scientists to productionise proactively developed analytics
  • Ensure that development is undertaken in line with continuous integration and test driven development philosophies and that code quality standards and processes are adhered to.
  • Work with Architects to both validate designs and ensure adherence to the agreed approach
  • Carry out unit testing to ensure the quality of delivered components
  • Must have

    We are looking for a Data Engineer with experience in Big Data processing technologies and techniques who is comfortable working alongside a strong, international team of engineers, to build applications for a key HSBC initiative.

    Whilst core skills are listed below, Bank is mainly looking for passionate people who are looking to continually improve and challenge themselves to work in a highly disciplined, verifiable manner.

    Core Skills / Characteristics

  • Broad understanding of Big Data technologies.
  • Experience with Hortonworks HDP / HDF
  • Experience with ELK stack
  • Experience with programming languages (Java, Scala)
  • Experience with messaging technologies (Kafka)
  • Experience with data processing frameworks (Apache Spark)
  • Nice to have

  • Experience of deploying services on Google Cloud Platform
  • Experience with programming laguages (Python)
  • Knowledge of DataFlow, BigTable, BigQuery, PubSub, Kubernetes, Airflow
  • Knowledge of machine learning libraries such as TensorFlow
  • Languages

    English : B2 Upper Intermediate

    Relocation package

    If needed, we can help you with relocation process. Click here for more information .


    deze vacature melden

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Mijn E-mail
    Door op "Doorgaan" te klikken, betekent dit dat je neuvoo toestemming geeft om je gegevens te verwerken en je e-mails met vacatures te sturen, zoals beschreven in neuvoo's -Privacybeleid . Je kunt je toestemming altijd intrekken