Originally published in 2018 in the IBM swift initiative blog website. Entire project initiative failed and all the articles were removed including this. Republishing the archive here just to keep my content alive. In 2021, this content is not so relevant.
At WWDC2017, Apple announced iOS 11 which has a new AI based framework, Core ML. Core ML helps the app developer bring iOS app development to the next level of enterprise business that demands AI and decision-making capabilities.
With server-side Swift’s ability to run on different OS platforms, including iOS, I did a quick PoC to explore and understand the Core ML capabilities. I integrated Kitura with iOS App and exploited the native iOS Core ML framework service as an open external API. In other words, Core ML is not a tightly bound service to use only with its built-in native iOS SDK. By integrating Kitura in the iOS App, you can make the service available for use with any external device. Now, a single iPhone device or an Xcode Simulator can serve as a mini server to provide the AI service to other devices.
The first part of this article is an introduction to Core ML. You will learn how to integrate Core ML with Kitura Server Side Swift. The second part of this article is a detailed comparison of Watson Visual Recognition and the Core ML service. This comparison should help you understand how enterprise ready these services are.
Core ML for iOS
Core ML is a framework that enables devices to run and process machine learning models. At the heart of the machine learning execution process is mlmodel, the trained model. Apple's developer documentation website includes links to some of the Core ML compatible format trained machine learning model files, for you to download and try.
Image Classification and Object Identification are interesting features of the deep learning concept. Its application and usage statistics in today’s enterprise development is also a key factor to consider for its gaining popularity. I created a simple iOS application that takes an image as input and uses a Core ML API and trained mlmodel file to recognize and classify the image. This app then converts to a hybrid Kitura Swift integrated application. For more information about this, read Kitura/iOS: Running a Web Server on your iPhone.
The idea to integrate server-side swift with iOS app is to expose the core ML service like any other standard REST API that accepts image data and returns the JSON result. The section below gives the solution to achieve this.
Delegation protocol and Server Swift multiple router handler
Delegation is a design pattern that enables a class or structure to hand off (or delegate) some of its responsibilities to an instance of another type. Delegation is one of the most commonly used patterns in iOS programming with protocol implementation. Server-side Swift router is designed to handle the request response with multiple handlers. Combining Delegation and the server-side Swift router gives the ability to divide the request handler tasks into subtasks and delegate any iOS specific subtask (API hits counting task, in this example) to the class instance methods. This combination is a quick workaround to extend iOS features as a Kitura Swift API service.
A few examples to think about include:
- API that does Core ML operations and returns JSON response
- API that sends email using iPhone device’s registered account
- API that returns reverse Geo Coordinates using Map Kit
- API that uses Siri's intelligence and Siri Kit to process request
In the code snippet below, the Kitura post request module uses three different handlers. The notifyRequest and responseProcessed methods delegate the task to the main view controller class for API request and responses notifications. processRequest calls the Core ML image processing module and returns the JSON result.
Code snippet: Kitura post request module uses three different handlers
mainRouter!.post("/data/analyzeImage",handler:self.notifyRequest,self.processRequest,self.responseProcessed)
func responseProcessed(request: RouterRequest, response: RouterResponse, next: @escaping () -> Void) {
DispatchQueue.main.async {
self.delegate!.didHitApi()
}
next()
}
func notifyRequest(request: RouterRequest, response: RouterResponse, next: @escaping () -> Void) {
DispatchQueue.main.async {
self.delegate!.didReceiveRequest(info: request.originalURL)
}
next()
}
func processRequest(request: RouterRequest, response: RouterResponse, next: @escaping () -> Void) {
var dataa = Data()
do {
let data = try request.read(into: &dataa);
…….
…..
Figure 1 shows the integrated app that runs the Core ML service and exposes the service using Kitura Swift. For every request, the API Hits count increases. The delegates in the main thread handle the API Hits. It's a success; there's a working prototype of a Kitura Core ML service running on an iOS device.
Read on to learn about the other iOS native only app to consume this service that I made. I've also included a comparison to the Watson Visual Recognition service.
Comparison of visual recognition services
Core ML is Apple’s iOS based framework to process machine learning models. Watson is IBM's platform built with AI and cognitive capabilities. Watson provides multiple AI and decision-making services for enterprise business systems and is available as a set of open APIs and SaaS products on Bluemix.
Curious about the accuracy and maturity of the systems, I extended the native-only PoC app to run and compare a set of images against:
- Core ML API made with Kitura Swift
- Uses the pre-trained VGG16 Keras Python model set
- Watson Visual Recognition Service API
This is a default API service which comes pre-configured with a trained example classifier engine to recognize images
I chose 15 different digitally processed images for the comparison. I chose processed images to test the accuracy of the trained model services when the input image loses natural color and pixel information. Because of file size issues, the images were further compressed to a JPEG format.
Figure 2 shows the app running with each service for comparisons. Two parameters of the service response are extracted in the screenshot, the Classified/Identified string, and the Confidence factor (the ranking is from 0-1).
Figure 2. The images and comparisons
Results
The result set below shows that Core ML with VGG16 identified 6 out of 15 images correctly. Of those six, two have a higher than 90% confidence index. Watson recognized 11 out of 15 images and had a 90% plus confidence index for seven of the images. There were some instances where the Core ML VGG16 model identified better. For example, the image of tablet pills on the right (shown in Figure 2, above). But, on a larger count, Watson predicted accurately more often. It is important to remember that the samples used are artificially processed and compressed images.
Figure 3. Results of the comparison
Conclusion
There's more to this than just the accuracy rate. Another consideration is the trained model. Both Watson and Core ML provide the ability to customize the classifier to use another model to improve accuracy. However, the effort to create and convert a better classifier trained ML model for iOS is much larger than using a simple Watson service. For Watson, you only need to provide a set of positive and negative images, and to train the engine. For Core ML you need to create a complex Python model and convert it into a mlmodel file.
The effort to create and convert a better classifier might with time. For now, though, Watson is a mature and stable service for the enterprise world. I consider Core ML to be an experimental platform until Apple releases significant improvements.
This article demonstrated how to convert any core iOS feature into a working Kitura Swift server-side service and compared the maturity and enterprise development readiness of Core ML and a Watson Service.
All the source code for this article is in my GIT repository, please fork and reuse. My next article will analyze the capabilities of converting the Core ML framework into a full-fledged server side framework… stay tuned.
References
- Build more intelligent apps with machine learning
- Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks
- What's new in iOS 11.0
- AVCapturePhoto
- Keras: The Python Deep Learning library
- Kitura/iOS: Running a Web Server on your iPhone
- Unlock the power of cognitive with Watson
- Bluemix
Disclaimer
This is not a peer nor expert reviewed article. I based the content on my knowledge and experience. My intent is to share my learnings and findings with fellow developers. The information might not be 100% accurate.
@Developers
The enterprise demands powerful and intuitive frameworks to transform applications to the next level of intelligence. It is important to learn, understand, and adopt the trending technologies early. Keep coding and keep learning. If you have any questions or comments, you can reach me at @sangesiv@in.ibm.
Happy Coding!!!!
No comments:
Post a Comment