Expert Details
Big Data Architecture: Research, Statistics and Software Engineering Solutions
ID: 739939
South Africa
Expert has extensive experience in advanced development methodologies, tools, and processes, contributing to designing and rolling out cutting-edge software applications using Big Data Technologies (Hadoop, NoSQL and homegrown). In this role, notable Big Data projects included Banking, Marketing Analysis, and Telecommunications using k-means algorithms. The latter maximized coverage of the telecommunication
company's users and increased customer satisfaction by up to 85%. I also have an innovative approach to developing solutions for clients to ensure the translation of the client's business requirements into specific
deliverables. Expert has an in-depth knowledge of research and statistics, and has several professional certifications, including Certified Big Data Hadoop and Spark Developer.
Skills: Hadoop/HDFS; Spark; Hive; Sqoop; Hbase; Flume; Pig; MapReduce; Java; Scala; Python; SparkML; Jupyter; Reactive Architecture; MongoDB; Kafka; Google Cloud Platform; Amazon Web Services (AWS); Azure Analytics Services; Databricks; Tableau; Linux; Git; Go
Education
| Year | Degree | Subject | Institution |
|---|---|---|---|
| Year: 2024 | Degree: MSc | Subject: Artificial Intelligence | Institution: University of Leeds |
Work History
| Years | Employer | Title | Department |
|---|---|---|---|
| Years: 2021 to Present | Employer: Undisclosed | Title: Big Data Architect | Department: |
Responsibilities:• Designed and fine-tuned design processes for implementing an Ontology Engineering pilot specific to the organization's data strategy and for the benefit of a critical quality business unit.• Directed the design cost-effectiveness for the pilot. • Overseeing the performance of data engineering workloads and making the appropriate recommendations on the self-serve platform and its interaction with the ingested data lake and other cloud services (Tools used include Spark, AWS Glue, Kafka, AWS Lake Formation, Azure Analytics Services and Azure Identity Services). • Led the design and digital twinning initiative to align the vision of product owners and the business objectives of use case owners. • Increased the collaboration and cohesion across numerous Ontology Layers using Monocle and other tools of Palantir Foundry. • Led presales technical support activity to prospective clients and stakeholders while ensuring customer satisfaction. • Coordinated different aspects of product analytics and technical trade-offs. • Advised internal and external clients on overall architectures. • Served as a strong communicator and refiner of processes across Business Analyst and Engineering sub-teams. • Liaised with multiple data stakeholders for the success of our Ontology. • Led a remarkable number of data governance, data modelling and standardization initiatives according to the management and organizational vision. • Contributed to improving the enterprise architecture principles through the Company's Architect community. |
|||
| Years | Employer | Title | Department |
| Years: 2017 to 2021 | Employer: Quantiflow Pty Limited | Title: Big Data Architect | Department: |
Responsibilities:• Set up data pipelines and architecture assessing the quality of a bank's marketing campaign for time deposit products.• Implemented various statistical models, Scala and Apache Spark. • Implemented a Hadoop risk modelling project to set up nodes, clusters, services and data pipelines for assessing bank clients based on their credit scores and determining the risk type, transaction similarity and likelihood of fraud. The tools and technologies used were Apache Hive and Python. • Set up the architecture and data pipelines for the CX project to determine the customer churn rate for a particular client product and possible causes. Tools and technologies: Scala, MongoDB and Pig. • Wrote Pig Latin scripts to determine the success rate of students from training datasets. • Analyzed a movie database using MapReduce, Java, Hive, Python and various user-defined functions. • Analyzed employee training scores to help a company know its employees' strengths and areas of improvement. Technologies used were MongoDB, NodeJS and Java. • Responsible for the Big Data Telecommunications project (k-means algorithm), implemented k-means clustering based on customer complaints and their respective locations and determined the areas of prospective mobile communication tower installations. Tools and technologies: Spark and Python • Facilitated the design and execution of project-based solutions, which maximized the coverage of the telecommunication company's users and increased customer satisfaction to up to 85%. |
|||
| Years | Employer | Title | Department |
| Years: 2015 to 2017 | Employer: ColonyHQ | Title: Software Engineer | Department: |
Responsibilities:Served as a Test Analyst for ColonyHQ's Brand Intelligence and Broadcast Engine.Responsibilities: • Addressed challenges and clarified issues relating to a quality assurance process for a broadcast aggregation web-based engine with more than 100 direct users and at least 20,000 daily interactions. • Incorporated automated test scripts using Selenium and Python for different application components on mobile and web platforms. • Worked at increasing the uptime and performance of the system to 98% using proactive test scripting, reworking user stories and client feedback—this reduced monthly clients' complaints rate from 70% to less than 5% on production. • Performance testing on different system components using LoadStorm and LoadImpact. • Successfully interacted with the team in enhancing various parts of the broadcast tool with relevant new user stories and added user-friendly features using Confluence and JIRA. • Assessed and documented risk analyses on test cycles before releases and identified defects, possible false positives and false negatives in various test scenarios; performed end-to-end white box, black box, user acceptance, regression, unit and manual levels of testing while giving prompt feedback to the different teams involved and product owner. The test cases also included verifying all components of data visualizations. |
|||
| Years | Employer | Title | Department |
| Years: 2013 to 2014 | Employer: ICT Worx | Title: Project Manager | Department: |
Responsibilities:• Provided the object-oriented analysis and object-oriented design documentation for a hospitality industry web-based application.• Wrote the scripts for the web page of an in-house software request banking application using HTML, CSS and JavaScript. • Worked closely with a support team to program the classes and interfaces in the Java Servlets Pages of the banking app using Java Enterprise Edition. • Assisted in developing the MySQL database for applications and deploying them to it. • Worked with Tomcat for servlet containment, Bootstrap, Hibernate and JQuery technologies. • Involved in the banking app project, which increased the efficiency of in-house software projects by 75%. • Interfaced with client requirements and developer teammates in identifying and formulating solutions for end-users. |
|||
| Years | Employer | Title | Department |
| Years: 2011 to 2013 | Employer: Animus Research Incorporated | Title: Research Associate | Department: |
Responsibilities:• Carried out due diligence financial research for clients and set up websites.• Assessed bottlenecks, gave possible solutions in customer relationship management, and provided statistical analyses for clients. |
|||
| Years | Employer | Title | Department |
| Years: 2008 to 2011 | Employer: Institute for Public Policy Research | Title: Research Associate (Statistics) | Department: |
Responsibilities:• Compiled and analyzed statistical data using modern and traditional methods to collect it, performed valid and reliable market research using SWOT analysis, interpreted data, formulated reports, made recommendations and used online research and catalogue findings to databases.• Provided competitive analysis on various companies' market offerings, identified market trends, pricing/business models, sales and methods of operation, and evaluated program methodology and critical data to ensure that data on the releases are accurate and the release angle is correct. • Liaised with relevant authorities at the National Bureau of Statistics (National Planning Commission, Windhoek) on up-to-date Namibian national data for the Institute for Public Policy Research website. • Set up different metadata for the 2009 Namibian Presidential and Local Elections for the Institute. • Reported on the elections from voting centres, live streaming analytics and analysis of the 2009 Namibian election results on the Institute's main election website. • Acquired and interpreted scientific results for Namibia's 6-Question Campaign on Access to Information. • Assessed needs and coordinated Namibian Climate Change Statistics for the Institute. • Prepared and presented scientific papers on platforms such as "Statistics for Climate Change Policies in Africa", Bonn, Germany and as an InWent/Economic Commission for Africa Expert in the "Acting on Climate Change for Sustainable Development in Africa: The 7th Africa Development Forum, United Nations Economic Commission for Africa, Addis Ababa, Ethiopia. |
|||
Career Accomplishments
| Licenses / Certifications |
|---|
| Certified Big Data Hadoop and Spark Developer, Big Data Architect Master Program | Big Data Hadoop and Spark Developer Program, (2017) Certified Kafka and Cassandra Developer, Big Data Hadoop and Spark Developer Program, (2017) Certified MongoDB Developer and Administrator, Big Data Hadoop and Spark Developer Program, (2017) National Certificate in Systems Development, Java Developer Academy, (2013 - 2014) Certificate in Climate Change Statistics, Statistics for Climate Change Policies in Sub-Sahara Africa, (2010) |
Additional Experience
| Training / Seminars |
|---|
| Data Engineering Path, Dataquest (2023) Fundamentals of the Databricks Lakehouse Platform, Databricks (2022) Data Foundations, Starburst (2023) Design Thinking, Social Innovation and Complex Systems, Project Management Institute (2023) Data Engineering and Data Principles, Lightbend (2020) Reactive Architecture: Advanced, Lightbend (2020) Reactive Architecture Foundations, Lightbend (2019) Elements of AI, University of Helsinki and Reaktor, Finland (2018) Data Science Foundations – Level 1, (2015) IBM CognitiveClass (formerly Big Data University) Hadoop Foundations – Level 1, (2015) IBM CognitiveClass (formerly Big Data University) Big Data Foundations – Level 1, (2015) IBM CognitiveClass (formerly Big Data University) |