Python Data Engineer
Confidential - Essentia Consulting recruting on behalf of
London, United Kingdom
Job TitlePython Data Engineer
Team / Department: The team belongs to the technology department and works closely with the business users across the businesses. The data engineer works in collaboration with software engineers, analytics teams, data scientists, and DBAs in order to understand and implement innovative analytics solutions across the business.
Team Overview / Purpose: The team is responsible for building and maintaining a Python-based platform for data analysis. This includes creating scalable and resilient data pipelines, developing new and better analytical capabilities and leveraging a range of technologies and predictive /statistical analysis methods.
Opportunity: This is an ideal role for an enthusiastic data engineer with strong Python development skills, and who wants to work in a greenfield project. The team is expanding its responsibilities across the organisation and scaling the existing data platform using cloud technologies. The data engineering team will innovate through exploration, design and benchmarking. The goal is to make technology recommendations and implement a scalable data analysis platform in the cloud. The role is also tasked with the development and implementation of a range of infrastructure and automation projects.
Key Responsibilities: Working with users to understand requirements as well as actively promoting the technology capabilities. The key responsibilities of the Data Analytics team are: 1. To build and maintain a scalable data platform. 2. Using the platform to build scalable data pipelines and ETL workflows. 3. Support other areas of the business with analysis, visualisations, automation, and performance bottlenecks.
Salary: To £90,000 (depending on experience) + benefits
- No telecommuting
- No Agencies Please
Behavioural Competencies Required:
- Demonstrates Integrity and Honesty
o Prioritises confidentiality and trust. Does not compromise on matters of ethics. o Is fair, consistent, open and honest in the treatment of others and his/her work. o Only gives commitments that can be met and ensures these are honoured. • Creative and Analytical Thinking o Ability to rationalise complex scenarios and promote collaborative solutions. o Confident to experiment and promote innovative solutions. o Able to define and explain problems clearly. • Interpersonal Skills o Ability to work effectively with different people and teams. o Acknowledges diverse opinions, addresses concerns, and promotes harmony. o Cooperates with others and works toward consensual solutions to achieve the group's objectives. o Effectively communicates and explains complex ideas.
Skills and Experience:
- Strong Python development skills including experience working with Numpy and Pandas.
- Understanding and experience working with data driven architectures.
- Proficient using Linux and Bash scripting.
- Experience using/building cloud-based solutions using AWS.
- Docker and Docker Compose.
- Experience working with efficient data storage formats such as Parquet, Avro or ORC.
- Management of virtual environments using anaconda and pip.
- Interest and willingness to teach and learn from others.
Beneficial Skills and Experience:
- Experience working with C/C++.
- Hands-on experience with Nifi, Airflow and Kubernetes.
- Familiarity with visualisation tools such as Plotly or Bokeh.
- Understanding of data platforms and resilient architectures.
About the Company
Innovative, rapidly growing, London-based technology company with sophisticated, proprietary data science platforms.
- Contact: Chiara Realini
- E-mail contact: firstname.lastname@example.org