Tags: Artificial Intelligence Jun 24, 2020, 22:06
Artificial Intelligence and Machine Learning Most Common BUzzwords in last 10 years but it is not new technology. it gained the popularity when the rise of deep learning, data revolution and availability of supported harware resources.In Current generation, most of companies use AI to handle the complex tasks in Different Sectors like Agriculture, e-commerce, Health Sciences, Deepfake, Market Analysis, Speech Recognition etc.
Normally We will Use Machine Learning Algorithms with different tech stacks like IT, Natural Language Processing, Computer Vision etc. And call this solutions as AI. Because this Type Solutions learn with Algorithms. We are using this type solutions in daily life Like netflix Recommendation Systems, Google Translate, Customer Assistent Chatbots, CT Scan in Radiology with AI, Weather Prediction, Stock Market Expectations etc.
But Here this AI solutions are very expensive to Develop at a time. Still Now, I Developed 4 Prototype AI Projects but in Deploy Time i got Many issues. Here We will discuss why AI Technology is Expensive..!
- Hard to Give the Estimates
- Skilled Developers
- Hardware Resource
- Handle the Different Situations Specially Accuracy
- Time Consumption
1.Hard to Give the Estimates
In traditional software projects we know everything what to do in next step so we can easily give the time estimates about project. but in AI Solutions it is really Hard to Estimates the Timings. Because it is dependent different Components like Data Quality, Understand the Data, Choose Best Algrithms, Hardware Requirements, Research Timing, Achieved Accuracy etc. so Giving the Estimates is big Problem in the project. my boss many times asked "please give the correct Estimates". but i don't know anything because only i can do after understand the data and the target goal. this is problem becasue it is not regular type software projects.
For your basic understanding i give a small problem. Imagine you have 100 Friends with images, contact details. when you upload one of friend image then display all his contact details in output screen. I know this problem work with compute vision but Here is My point Please Calculate the Time Estimates. (note: some of friends not have correct names and contact details).
I am Learning and Working AI from 3 years with different tech stacks like Machine Learning Algorithms (include deep learning), Natural Language Processing, Computer Vision, IoT technologies. But Still now i don't know some techniques.companies need High skilled developers with an Experience. but not many developers are highly skilled. here is my view two sides. skilled developers are very expensive developers but others are also improving their skills. right..! so remember two sentences here why AI is Expensive.
- you can't paid high salary to correct AI developers (especially in startup companies).
- Companies need High Skilled Ai developers but they are less in industry.
In Last 15 years back, we don't have minimum Hardware Resources to Run the Machine Learning Algorithms. but after rise of Internet, the high amount data produced every day. so we are developed Powerful Hardware Resource. now we can easily run machine learning algotithms in current resources. but the hardware requiremtns are very expensive. but why..!
Because Data is in Different Formats and the Data Amount is Big. yes..! I recently developed a application with image data. the application idea is simple "find the all related images of given image". when i train the data with deep convolutional networks the training time taken is too much with the CPU and minimal RAM. so i need better system with Dedicated GPU. i was search in online to the physical systems. the physical systems cost is very high ($4000+) and maintainance is extra. so i decided use the cloud computing technology to my application but the monthly costs of machine less cheap (280$). so hardware requirements cost are very high when you want develop/deploy the AI Applications. I Hope this will reduce in future with nano/quantum technology.
4.Handle the Different Situations Specially Accuracy
The most important parameter is Accuracy it is depedent on train/test data, hyper parameter tuning, data quality and Choosen Algorithm. So developers use Diferent Algorithms and tecchnques to Achieve the Good Accuracy in Training/Valiadation Steps. if Accutacy is the Good then we don't worry about the predictions. but the problems are handle the Difficult Situations in Machine Development Life Cycle. I really Dissapointed when 3 years back because i trained everytime with different algorithms and wait to the results. now i know Better Method MLOPS to handle this situation. Thanks to My Gurus to teach this in Cloud Platforms. so this is most Important To Handle the Product Release in Correct Time. only skilled persons handle this situations. so the final solution is very expensive to habdle this in the project.
In Machine Learning Life Cycle, Data Play a Very Important role in all stages. mostly in data preprocessing Stage. Because Data is Messy and It is in Different Formats. I rellay struggled to create training date set before Understadn the data like Text and images. It is Kicked in my hed everytime and i have less experience in Date Preprocessing. So i leaened everything about data with data science (not much in business level). so time is going in the project too high, generally like this.
- Gather the Data - 5%
- Data Preparation - 20%
- Analyse the Data - 15%
- Train the Data - 35%
- Test / Evolution - 15%
- Deployment - 10%
The project Cost go to High when Poject time Increase Continuosly. Look the Data is Taken almost 40% time in first 3 steps and Data Training is in Second position. But if We use Distribution Computing Techniques the Deployment Stage takes the over time in the project.
so this five aspects make the product cost to very high in Artificial Intelligence Solution. I Hope You Understand This when you give some Machine Learning project to Your Developer.