Description
As a data platform solution engineer you act as an enabler for data engineers developing AI, BI and data integration use cases on the enterprise data platform. You support teams in the use of the data platform, e.g. you assist in setting up new projects or help set and uphold best-practices, standards and guidelines. Your goal is to improve the user experience and easy the adoption of the data product architecture by these use case teams.
On the other hand, you participate in the ongoing data platformization effort. Driven by the data platform roadmap and data architecture vision, you design and implement best practices, integrations, libraries and frameworks realizing new or enhanced data platform capabilities that improve the data products observability and contribute to computational data (product) governance. As such you are in the forefront of the efforts to increase the enterprise data maturity.
Way of working keywords: Agile principles, continuous improvement, peer reviewing / pull requests, clean code, testing, … Good listening and communication skills (can talk both with business as with IT stakeholders).
Responsibilities
- Enable data engineers in the use case development and use of new data platform capabilities.
- Create technical designs and build enabling features, libraries, blueprints.
- Define development standards, guidelines & best practices.
- Develop and maintain sample projects, blueprints, documentation, …
- Assist in the intake for new features.
- Assist in planning & providing high-level estimates for new development.
- Develop proof-of-concepts.
- Help with proposals and (technical) requirements gathering, RFP’s
Skills & Experience
Seniority in technical designs, software engineering and data engineering. Experienced in software framework & library development and (release) management. Value-driven, thorough understanding of agile software delivery.
Pragmatic thinker, strong analytical skills. Strong people, coaching, communication and presentation skills. Fluent in English (verbal and in writing), Dutch or French optional.
Hands-on, proven, senior experience with (“the basics”):
- SQL
- Python & PySpark
- Snowflake
- Airflow
- JSON, Parquet
- Building data pipelines with Dbt
- Data ingestion (e.g. CDC, API, …)
- Event-based data processing
- Data processing in an AWS cloud environment: AWS services (S3, Glue, Athena, AppFlow, DMS, …) & IAM
- IaC, Terraform
- Agile delivery, CI/CD, Git
Experience with or profound interest in (“nice to have”):
- Data product architecture
- Data warehousing, data/delta lake architecture
- Observability, monitoring, alerting
- Apache Kafka
- Apache Iceberg
- AWS Kinesis, SQS, Lambda, Sagemaker, Bedrock
- Data quality metrics, SLI/SLO/SLA
- Data quality, data catalogue, data lineage tooling
