{"id":2404,"date":"2021-08-31T12:58:15","date_gmt":"2021-08-31T16:58:15","guid":{"rendered":"https:\/\/www.day5analytics.com\/?p=2404"},"modified":"2026-08-17T08:24:27","modified_gmt":"2026-08-17T08:24:27","slug":"deploying-and-controlling-intelligent-predictions-for-financial-performance-part-1","status":"publish","type":"post","link":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/2021\/08\/31\/deploying-and-controlling-intelligent-predictions-for-financial-performance-part-1\/","title":{"rendered":"Deploying and Controlling Intelligent Predictions for Financial Performance (Part 1)"},"content":{"rendered":"<p><a href=\"http:\/\/www.day5analytics.com\/\"><img fetchpriority=\"high\" decoding=\"async\" class=\"wp-image-3789 size-full alignright\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/newImg81.webp\" alt=\"\" width=\"325\" height=\"575\" srcset=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/newImg81.webp 325w, https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/newImg81-170x300.webp 170w\" sizes=\"(max-width: 325px) 100vw, 325px\" \/><\/a><\/p>\n<p><a href=\"http:\/\/www.day5analytics.com\/\">Day5 Analytics<\/a>\u2019 client completes thousands of residential electrical, HVAC and plumbing service requests each month \u2013 where technicians attend a problem and attempt to repair it. The cost of each request is affected by various factors \u2013 such as, when the request is made, the urgency with which work needs to be completed, the number of previous calls made to service the same equipment, etc. The final cost of the service is only known when work is finally completed \u2013 however, the client needs an accurate cost estimate to manage cashflow, inform discretionary spending, and assess whether there is value in delivering the repair service. Basic cost estimation methods result in crude high-level estimates, that lead to missed financial opportunities, and challenge decision making at the field level.<\/p>\n<p>Using a combination of automatic machine learning, simplified field deployable applications and an automated way to ensure predictions remain accurate, Day5 Analytics developed a solution that drives cost estimate improvements of 40%.<\/p>\n<p>Over a 3-part blog series, Day5 Analytics demonstrates how to create and deploy a machine learning solution with an innovative blend of low-code and low-cost technologies.<\/p>\n<ul>\n<li>Part 1 will focus on the production deployment of cost prediction using automated machine learning, using <a href=\"https:\/\/www.knime.com\/\">KNIME<\/a> and <a href=\"http:\/\/www.pycaret.org\/\">PyCaret<\/a><\/li>\n<li>Part 2 will demonstrate how to allow field operators and various systems to leverage the cost predictor via a simple user interface, using <a href=\"https:\/\/powerapps.microsoft.com\/en-ca\/\">Microsoft PowerApps<\/a> and <a href=\"https:\/\/www.knime.com\/knime-server\">KNIME Server<\/a><\/li>\n<li>Part 3 will explore how deployed prediction models can be monitored and automatically retrained for accuracy when drift is detected, using KNIME Server<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<h2>Part 1: Improving Cost Forecasts with PyCaret Automated Machine Learning in KNIME<\/h2>\n<p><a href=\"https:\/\/www.knime.com\/knime-analytics-platform\">KNIME<br \/>\nAnalytics Platform<\/a> is an open-source software program for creating data<br \/>\nscience workflows. <a href=\"https:\/\/www.knime.com\/knime-server\">KNIME Server<\/a><br \/>\nis used to deploy the workflows as analytical applications and services.<br \/>\nWorkflows developed in KNIME are easy to setup and understand, with no coding<br \/>\nrequired. KNIME\u2019s ease-of-use accelerates time-to-value, and as an open-source<br \/>\nplatform, integrations with a growing number of tools and software allow for near<br \/>\nunlimited flexibility and problem-solving coverage.<\/p>\n<p><a href=\"https:\/\/pycaret.org\/\">PyCaret<\/a> is an automated<br \/>\nmachine learning library, that performs data preparation and machine learning<br \/>\nmodelling within minutes. With KNIME\u2019s proprietary <a href=\"https:\/\/www.knime.com\/integrated-deployment\">Integrated Deployment<\/a> functionality,<br \/>\nusers can move from testing to deployment without additional changes or manual rework<br \/>\nrequired \u2013 which is typically where deployment of data science applications<br \/>\nfalter. This accelerates the time to production, reduces errors during transfer<br \/>\nfrom development to production, increases compliance, and results in a streamlined<br \/>\nand automated processes.<\/p>\n<h3>Model Training and predictions<\/h3>\n<p>Data sourced from an ERP system is consolidated and preprocessed in KNIME for machine learning. Any data preparation steps that need to be repeated for predictions is contained within a pair of <strong>Integrated Deployment<\/strong> nodes (Capture Workflow <a href=\"https:\/\/hub.knime.com\/knime\/extensions\/org.knime.features.buildworkflows\/latest\/org.knime.buildworkflows.capture.start.CaptureWorkflowStartNodeFactory\">Start<\/a> and <a href=\"https:\/\/hub.knime.com\/knime\/extensions\/org.knime.features.buildworkflows\/latest\/org.knime.buildworkflows.capture.end.CaptureWorkflowEndNodeFactory\">End<\/a>); this generates a deployment-ready workflow. The dataset is then split into training and testing sets for machine learning with PyCaret.<\/p>\n<p>PyCaret allows both regression and classification automated machine learning techniques in\u00a0 low-code fashion, among other modelling methods. The installation of PyCaret in KNIME can be found <a href=\"https:\/\/medium.com\/low-code-for-advanced-data-science\/machine-learning-in-knime-with-pycaret-420346e133e2\">here<\/a>.<\/p>\n<p>Training data is passed to PyCaret embedded within KNIME for predictive modelling. PyCaret tests various models on the dataset, and returns the best performing model.\u00a0This model is saved locally and loaded into a Python Predictor node, which is assessed for accuracy against test (unseen) data. The prediction steps are also captured with Integrated Deployment nodes.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/workflow_preprocessing.png\" sizes=\"(max-width: 569px) 100vw, 569px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_preprocessing.png?w=569&amp;ssl=1 569w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_preprocessing.png?resize=300%2C132&amp;ssl=1 300w\" alt=\"\" width=\"569\" height=\"251\" \/><br \/>\n<img decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/workflow_model_predictions.png\" sizes=\"(max-width: 413px) 100vw, 413px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_model_predictions.png?w=413&amp;ssl=1 413w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_model_predictions.png?resize=300%2C180&amp;ssl=1 300w\" alt=\"\" width=\"413\" height=\"248\" \/><\/p>\n<h3>Model Deployment<\/h3>\n<p>The deployment-ready machine learning model is consolidated and written to a workflow, for which KNIME generates a corresponding REST API through its Swagger interface. The workflow is then deployed to the KNIME Server, and can receive new data for predicting cost.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/workflow_rest_api.png\" sizes=\"(max-width: 624px) 100vw, 624px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_rest_api.png?w=624&amp;ssl=1 624w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_rest_api.png?resize=600%2C185&amp;ssl=1 600w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/workflow_rest_api.png?resize=300%2C92&amp;ssl=1 300w\" alt=\"\" width=\"624\" height=\"192\" \/><\/p>\n<p>A third workflow (Remote Workflow Call) is used to call the deployment workflow. Data is received via REST API, and passed to the deployment workflow via KNIME\u2019s Call Workflow functionality. The deployment workflow produces a cost prediction, which is passed back to the Remote Workflow Call workflow, which in turn passes the cost prediction back to the mobile application via REST API.<\/p>\n<p>API enables integration with various third-party systems and the client\u2019s own internal systems (without major integration investments), allowing the client to accurately predict cost as soon as a request is received. This is covered in Part 2 of the blog series.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/remote_workflow_call.png\" sizes=\"(max-width: 459px) 100vw, 459px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/remote_workflow_call.png?w=459&amp;ssl=1 459w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/remote_workflow_call.png?resize=300%2C91&amp;ssl=1 300w\" alt=\"\" width=\"459\" height=\"139\" \/><\/p>\n<h3>Cost Prediction Accuracy<\/h3>\n<p>The cost of a service request can range from tens to thousands of dollars. Prediction error is assessed using the <strong>Mean Absolute Error (MAE)<\/strong> \u2013 the absolute difference between actual and predicted cost, averaged over all predictions. In all cases, the machine learning model is able to predict costs closer to the real amounts, by $140 or more, to drives business cost certainty.<\/p>\n<p>In our sample, the MAE with PyCaret is improved by 38% compared to the simple average (status quo) method; a significant improvement. This occurs not only at an aggregate level, but even when compared across months and services \u2013 indicating that the performance improvement can be leveraged at deeper levels of the operation.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement_Aggregate.png\" sizes=\"(max-width: 800px) 100vw, 800px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement_Aggregate.png?w=947&amp;ssl=1 947w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement_Aggregate.png?resize=600%2C172&amp;ssl=1 600w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement_Aggregate.png?resize=300%2C86&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement_Aggregate.png?resize=768%2C220&amp;ssl=1 768w\" alt=\"\" width=\"800\" height=\"229\" \/><br \/>\n<img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement.png\" sizes=\"(max-width: 486px) 100vw, 486px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement.png?w=486&amp;ssl=1 486w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction_Error_Improvement.png?resize=300%2C78&amp;ssl=1 300w\" alt=\"\" width=\"486\" height=\"126\" \/><\/p>\n<p>Plotting the distribution of errors for each model allows for a better understanding of error reduction \u2013 the PyCaret error distribution skews closer to the real cost than the status quo (average) method.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE-768x388.png\" sizes=\"(max-width: 768px) 100vw, 768px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE.png?resize=768%2C388&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE.png?resize=600%2C303&amp;ssl=1 600w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE.png?resize=300%2C151&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE.png?resize=1024%2C517&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/Prediction-Improvement-MAE.png?w=1109&amp;ssl=1 1109w\" alt=\"\" width=\"768\" height=\"388\" \/><\/p>\n<p>The model also provides business insight into <strong>drivers<\/strong> that affect service request cost, using a <strong>feature importance plot<\/strong>.<\/p>\n<p>The \u2018previous call\u2019 feature and the \u2018day of the service call\u2019 (<em>i.e., how many service requests of the same category were received in the prior 6 months<\/em>, and <em>weekday\/weekend \/statutory holiday respectively<\/em>) impact the cost most significantly.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-content\/uploads\/2021\/08\/feature_importance.png\" sizes=\"(max-width: 480px) 100vw, 480px\" srcset=\"https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/feature_importance.png?w=480&amp;ssl=1 480w, https:\/\/i0.wp.com\/www.day5analytics.com\/wp-content\/uploads\/2021\/08\/feature_importance.png?resize=300%2C188&amp;ssl=1 300w\" alt=\"\" width=\"480\" height=\"301\" \/><\/p>\n<h3>Next Steps<\/h3>\n<p><em>The prediction KNIME model can be found on the KNIME Hub, <\/em><a href=\"https:\/\/kni.me\/w\/u53goeMdMZx_x-p_\"><em>here<\/em><\/a><em>.<\/em><\/p>\n<p>Part 1 of the blog post series detailed the development, assessment, and deployment of the machine learning model cost predictions compared to the \u2018status quo\u2019 average cost method. Part 2 and Part 3 will demonstrate the Microsoft Power Apps mobile application for field deployment, and model monitoring and automatic retraining functionality built in KNIME Server, respectively.<\/p>\n<p>To inquire about custom-built and deployed machine learning solutions at your company, contact <a href=\"http:\/\/www.day5analytics.com\">Day5 Analytics<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Over a 3-part blog series, Day5 Analytics demonstrates how to create and deploy a machine learning solution with an innovative blend of low-code and low-cost technologies.<\/p>\n","protected":false},"author":1,"featured_media":2437,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_angie_page":false,"page_builder":"","footnotes":""},"categories":[68],"tags":[50,51,74],"class_list":["post-2404","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-solutions","tag-finance","tag-knime","tag-machine-learning"],"_links":{"self":[{"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/posts\/2404","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/comments?post=2404"}],"version-history":[{"count":4,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/posts\/2404\/revisions"}],"predecessor-version":[{"id":3800,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/posts\/2404\/revisions\/3800"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/media\/2437"}],"wp:attachment":[{"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/media?parent=2404"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/categories?post=2404"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mindblowpreviews.com\/stagings\/day5-analytics\/wp-json\/wp\/v2\/tags?post=2404"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}