Apple Intelligence: Here’s what you can do with Apple’s Generative AI

KUALA LUMPUR, June 11 — During Apple’s WWDC 2024 keynote, Apple Intelligence is the biggest announcement besides the latest operating systems. This is Apple’s AI for iPhone, iPad and Mac which aims to deliver a new personalised experience while ensuring privacy and security. In a way, it is Cupertino’s answer to Samsung’s Galaxy AI as it brings new Generative AI features to help simplify your day-to-day tasks.

According to Apple CEO Tim Cook, Apple Intelligence marks a new chapter in Apple Innovation and it will transform what users can do with their products and what their products can do for their users.

He added, “Our unique approach combines generative AI with a user’s personal context to deliver truly helpful intelligence. And it can access that information in a completely private and secure way to help users do the things that matter most to them. This is AI as only Apple can deliver it, and we can’t wait for users to experience what it can do.”

AI-assisted Writing Tools

One of Apple Intelligence’s biggest capabilities is helping users with writing and communications. With its new Writing Tools built into iOS 18, iPadOS 18 and macOS Sequoia, users can get help to summarise, rewrite and proofread their text in emails, notes, messages and other third-party applications including instant messaging apps or even blog posts. For example, if you’re writing an email, you can use Writing Tools to rewrite the content based on the desired tone to be professional or friendly.

Writing Tools can also assist you if you need to summarise or highlight key pointers of the message. Just highlight the desired text and choose to recap in either a paragraph, bulleted keypoints, a table or even a list.

Improved Siri with better context awareness

With Apple Intelligence, you can expect Siri to deliver a more natural, relevant and personalised experience. It is more deeply integrated and it even works if you stumble over words and it can continue to maintain the context from one request to the other. Even if you need help or want to activate a feature without knowing the exact name, Siri can accommodate such requests by acting like a Chatbot for the iPhone user manual.

If you prefer not to speak, there’s now an option for you to type to Siri which is great when you’re stuck in a meeting or in an environment where talking to Siri isn’t feasible.

To better understand and provide the right action, Siri now has on-screen awareness. For example, if someone shares an address over text message, you can simply say “Add this address to his contact card”. Or if you want to share what you’re viewing with someone, you can simply say “Add this photo to the email I drafted to John”, or “Send this article to Sophie”.

GenMoji and AI-generated images at your fingertips

Apple is also bringing AI-generating images to its devices via Apple Intelligence. With GenMoji, there are endless emoji possibilities as you can generate anything you want based on your desired description. Since this is baked-in system-wide, you can add Genmojis practically anywhere as an emoji, sticker or even as a reaction in a Tapback.

To help communicate or express yourself better, you can also create your AI-generated image with Image Playground. This is essentially a dedicated AI-generated app which is also built-in into apps such as Messages. Besides mixing and matching different subjects and objects, you can also choose your own style such as Animation, Illustration or Sketch. These images are generated on-device and you can create and experiment with as many images as you want.

If you have an iPad with an Apple Pencil, there’s a feature called Image Wand in Notes, which turns your rough sketches into professional-looking AI art. From a simple sketch, you can provide more descriptions and the Image Wand will create an image based on the context that you provide. In a way, this is similar to Microsoft’s Image Creator in Paint for AI-generated graphics.

AI-powered Clean Up tool for Photos

Like most AI-powered smartphone these days, Apple has also introduced its AI-removal tool called Clean Up that’s baked into the Photos app. With a simple tap or drag, Apple can use AI to remove unwanted objects in the background without affecting your subject.

Find the photos or video moments you want is now more powerful with Apple Intelligence. Now you can find specific images or videos with detailed searches such as “Find Maya skateboarding in a tie-dye shirt” or “Katie with stickers on her face”.

What’s even more interesting is you can now create your own custom “Memory” video in the photos app based on your provided description. Using language and image understanding, it will pick the best photos and videos based on your description and generate a video based on your storyline while suggesting a matching background song from Apple Music.

Prioritised notifications

With more apps and messaging platforms on your device, it can be hard to keep track of what really matters to you. With Priority Messages, your inbox and even the iPhone lock screen can filter down to show only the most urgent emails and messages. The feature puts important matters such as same-day dinner appointment or your flight’s boarding pass upfront.

If you receive tons of emails in a day, you can also view summaries of each message without opening individual emails or threads. The same also applies to group messages in the Message app so that you can get up to speed of what’s new at a glance.

To help you respond quickly, there’s also Smart Reply which provides suggestions automatically based on context. By answering a few questions with a few taps, the feature will automatically draft a response for you.

Apple Intelligence emphasises privacy and security

Apple says they are setting a new standard for Privacy in AI as Apple Intelligence mainly runs on on-device processing and many of the models that power it run entirely on the device. However, if there’s a need for more processing power for AI, Apple taps on its Private Cloud Compute which scales its computational capacity via the cloud powered by Apple Silicon servers.

It added that the models that run on the servers provide a foundation that allows Apple to ensure that data is never retained to exposed. Apple says its privacy promise can be verified and they allow independent experts to inspect the code that runs on their Apple Silicon servers to verify its claims.

Seamless ChatGPT integration

To expand its AI possibilities, Apple has also announced ChatGPT integration across its platforms via iOS 18, iPadOS 18 and macOS Sequoia. Users can immediately tap on ChatGPT’s image and document understanding capabilities without jumping between tools as it is available systemwide through Writing Tools.

Siri can also tap into ChatGPT’s expertise when required and users can also use ChatGPT image tools to generate images to complement their content.

With security and privacy in mind, users must grant permission to allow sharing with ChatGPT. With the integration by Apple, users’ IP addresses are obscured and OpenAI won’t store the requests. Users will be able to access ChatGPT powered by GPT-4o for free without creating an account. However, paying users can connect their accounts and access the paid features on Apple devices.

What devices support Apple Intelligence

During the keynote, Apple revealed that Apple Intelligence requires a lot of processing power to handle on-device Generative AI. At the moment, it only supports iPhones running on A17 Pro which is essentially the iPhone 15 Pro and iPhone 15 Pro Max. For iPad and Macs, it is supported on devices running on Apple M1, M2, M3 and M4.

Apple Intelligence Beta is rolling out in Fall (sometime in September) and it will be free to use and available only in US English. You can check out the preview on Apple’s website. — SoyaCincau