Automatically optimize the execution of your pipeline on available infrastructure resources
Sep 3, 2020 from Caroline Pacheco
Let’s suppose that you have a large infrastructure containing several machines that have different operating systems (e.g. Microsoft Windows, Linux, MacOS) and distinct hardware configurations, such as Processor (CPU), Video Card (GPU), Memory (RAM) , Storage (Hard Drives) and others. Furthermore, some of these machines are available in AWS, Azure or Google Cloud. Now, let’s suppose you want to run several machine learning algorithms in parallel using this hybrid infrastructure, how could you do it in an optimized way?
As you can imagine, this is not a trivial task due to time consumption and the requirement of specific technical knowledge. For this purpose, Activeeon has released the latest version of its Machine Learning Open Studio (MLOS). It provides a flexible solution for users to distribute and parallelize a variety of artificial intelligence (AI) workloads (machine learning, deep learning, computer vision, etc) on a large infrastructure and to leverage hybrid and multi-cloud capabilities. MLOS helps data scientists and IT operations work together in an MLOps approach allowing to bring easily ready-to-be-used ML models to production. MLOS also simplifies machine learning application lifecycle management providing an end-to-end orchestration, automation and scalability.
To minimize the complexity of AI with repeatable & scalable machine learning lifecycle, MLOS provides powerful catalogs with some examples of machine learning & deep learning tasks & workflows that are ready to run. Nevertheless, it’s open from end to end and you can modify or adapt everything on it. You can also implement your own machine learning algorithm in different programming languages, such as Python, R, Java, etc. In addition, you can use the AI framework/library you prefer, such as Sklearn, Torch, TensorFlow or Keras.
In the Figure below you can see three end-to-end machine learning pipelines built by drag and drop to train three algorithms on a large infrastructure. In this example we use three algorithms: Logistic Regression, Support Vector Machine and Random Forest to predict vehicle type (e.g., Opel, Saab, Bus, Van) based on silhouette measurements.
If you don’t prefer the drag and drop solution — don’t worry. MLOS also allows you to deploy your machine learning pipeline using your favorite tool through our Python/R/Matlab SDK. You can do your ML pipelines through deployment as well using the Jupyter Lab instances by our Proactive Jupyter Kernel. Our platform will automatically optimize the execution of your pipeline on available infrastructure resources. If you have any questions or feedback, feel free to send us an email to firstname.lastname@example.org. Our team would be very pleased to get your feedback or help you in any way possible.
Sep 3, 2020 from Caroline Pacheco
Let’s suppose that you have a large infrastructure containing several machines that have different operating systems (e.g. Microsoft Windows, Linux, MacOS) and distinct hardware configurations...
Mar 13, 2020 from Activeeon
Users can interact with third-party systems in two ways: using ProActive web portals (Studio, Automation Dashboard, Scheduler, Resource Manager) or using APIs (REST, Java, CLI, etc.)...
Dec 5, 2019 from Activeeon
A job scheduler executes workloads based on a certain scheduling policy. An advanced job scheduling solution can support several scheduling policies that determine how jobs and tasks will be scheduled. These include First-In-First-Out (FIFO), Earliest deadline first (EDF), and License-based policies....