Apple Intelligence AI, iOS 18 and the biggest announcements at WWDC 2024

Yesterday’s Apple’s Worldwide Developers Conference keynote teased a lot of what users can expect this fall when big iOS, iPadOS, macOS and watchOS updates hit their devices. Changes coming include RCS support in Messages, a new Passwords app, a revamped Calculator app for iPhone and iPad and a bunch of artificial intelligence (AI) infusions across the board with the new “Apple Intelligence” system. The latter will bring some of the biggest updates to Apple devices in years, including generative AI image creation, “Genmoji” custom emojis, text summarization and even some ChatGPT integration as well. If you weren’t able to catch the news live, here’s a rundown of everything announced at WWDC 2024.

Apple Intelligence

Apple revealed its plans to incorporate AI into its operating systems at WWDC this year. Dubbed “Apple Intelligence,” this new generative AI system will appear in iOS and iPad 18 and macOS Sequoia in the form of (what Apple believes to be) practical tools that most people can use regularly. Those features include new writing tools that can help you rewrite, proofread and summarize things like emails and other messages, original emoji and image creation and more. Going hand-in-hand with original image generation is a new feature called Genmoji, which allows users to create their own unique emojis by typing in descriptions and requirements like “T-rex wearing a tutu on a surfboard.”

Siri is getting an AI infusion, now that it will be powered in part by large language models. In addition to asking Siri to delete an email or edit a photo, users will also be able to ask the virtual assistant to summarize articles and webpages in Safari and even extract personal information from a picture of an ID so it can fill out an online form for them. The company emphasized the importance of “personal context” with Apple Intelligence, which will enable things like using natural language to search for photos that contain only specific family members or friends.

Apple highlighted how most Apple Intelligence actions will be done on-device to make the system as privacy-focused as possible. For queries that cannot be done locally, the work will be sent to Apple’s processing centers. The company also created Private Cloud Compute, a feature that’s supposed to utilize the cloud for more advanced AI processing while also making sure your data remains secure.

OpenAI’s ChatGPT is also integrated into Apple Intelligence, allowing users to give Apple permission to share their queries with ChatGPT “when it might be helpful.” Examples provided include asking for menu ideas that incorporate specific ingredients, or asking for decor advice while providing a photo of a space that needs sprucing up. ChatGPT will also work with the AI writing tools coming to iOS and iPadOS 18 in a new Compose feature. ChatGPT integration with iOS 18, iPadOS 18 and macOS Sequoia will roll out later this year, and apparently Apple intends to add support for other AI models in the future — meaning its partnership with OpenAI isn’t a long-term exclusive.

iOS 18 and iPadOS 18

The next iPhone software update will roll out to users in the fall and, as expected, one of the biggest changes is support for Rich Communication Service, or RCS. The messaging protocol offers many improvements over SMS including end-to-end encryption, better media sharing and support for proper group chats. Apple previously stated it would adopt support for RCS in 2024 to comply with EU regulations, so it’s unsurprising to see it mentioned in iOS 18’s forthcoming features. Also new to Messages will be the ability to “tapback” reply using emojis and stickers, text formatting and effects and the ability to send messages via satellite.

iPhone users will have more control over their home screens in iOS 18 thanks to the fact that it will not be a locked grid system anymore. Users will be able to move app icons more freely, plus they’ll be able to change app icon colors as well use a tint color picker. In terms of design and layout, this is one of the biggest changes to come to the iPhone’s home screen in years and it gives iOS users similar features to those Android users have had for a long time. In the same vein, Control Center will be updated in iOS 18 to include more customization options, and will allow users to program quick controls from third-party apps in addition to the native options.

The Photos app is getting a big redesign in iOS 18, putting an emphasis on intelligently organized groups of photos that revolve around memories, trips and other big events. The new design ditches the old tabbed layout and will usher in a one-page design when you can view all of your photos individually, or view them by Collections. Users will also be able to filter out things like screenshots and receipts that would show up in a chronological format, but would otherwise mess up a tightly curated group of vacation photos.

A couple of new privacy features stand out in iOS 18, namely the ability to lock and hide apps. For the former, users can lock an app so sensitive information stays behind a Face-ID or Touch-ID wall, preventing those who you casually hand your iPhone to from seeing that information. Hiding an app, on the other hand, does exactly what you think: hides a program in a special hidden folder that others won’t be able to see.

The Calculator app is getting a big overhaul in iOS 18, including improved unit conversions, a sidebar showing recent activity and integration with the Notes app. But what might be even more notable is the fact that the revamped Calculator app will not only be available on iPhones and Macs — it’s coming to iPads for the first time as part of the iPadOS 18 update. Embedded within the iPadOS Calculator app is a new feature called Math Notes, which lets users write out math equations with the Apple Pencil and the app will solve many of them instantly.

iPadOS 18 will also feature a new Tab Bar, which looks similar to the Dynamic Island on iPhones. This bar makes it easier to access essential controls even when you’re in apps, and depending on what you’re doing, it can show up at the top of the screen or as a sidebar of sorts on the left of the display. The Notes app in iPadOS is getting another new feature called Smart Script, which will make users’ handwriting more legible automatically.

macOS Sequoia

The next iteration of Apple’s computer software will be called macOS Sequoia. In addition to many of the AI features also coming to iOS and iPadOS 18 as part of Apple Intelligence, the next macOS update will include iPhone mirroring, which lets users see and control their iPhone screen on a Mac screen. They’ll be able to use their keyboard and trackpad to intact with the iPhone screen on their laptop, and they can even open iOS apps directly on their computers without picking up their iPhone at all.

A new Passwords app builds upon the technology of iCloud Keychain to save all of users’ passwords and login credentials across devices and platforms (it will be available on Windows in addition to iOS and iPadOS). Along with standard passwords, the new app can save passkeys, verification codes and more, and give users the ability to securely share passwords with others.

Other updates coming in macOS Sequoia include a snap window arrangement tool with accompanying keyboard and menu shortcuts, Presenter Preview, which lets you see what you’re about to share with call partners before they see it, and gaming upgrades like improved Windows porting capabilities using Gameporting Toolkit 2. Users will also get access to Image Playground in macOS Sequoia, Apple’s AI image generator built into Apple Intelligence. It provides the ability to create AI-generated images in different styles, including animation, illustration and sketch.

watchOS 11

The next software update for the Apple Watch includes two big changes: Training Load and a new Vitals app. Training Load in watchOS 11 essentially uses many of the health and fitness metrics collected during workout tracking to estimate your effort level each time. Each workout will receive a rating from one (easy) to 10 (all out) that estimates how hard the user worked during that particular session.

The new Vitals app will show Apple Watch users how their captured health data, including heart rate, compares to baseline measurements. This will hopefully allow users to better understand when something might be off and outside the “normal” range.

The Activity app on iPhone is also getting an update to accompany watchOS 11, and will allow users to customize the data they see on the homepage so they can put their most important stats front and center. Cycle Tracking will also get an update to include more detailed pregnancy insights, including gestational age and information about the user’s health metrics that may related to pregnancy (like heart rate fluctuations).

visionOS 2

Until now, Apple’s Vision Pro headset has only been available in the US. That’s changing soon as the company announced the device’s rollout in additional countries including Australia, Canada, China, France, Germany, Japan, Singapore and the UK in the coming months. As far as the headset’s software (visionOS) goes, Apple announced that visionOS 2 will add spatial photos, which adds depth to images in the Photos app, new UI gesture controls and improved Mac screen mirroring with support for higher resolutions and display sizes.

AirPods Pro audio updates

Apple briefly mentioned some software updates coming to AirPods Pro, including improved Voice Isolation, which should help the buds better pick up a user’s voice in noise environments. A new Siri Interaction is coming to AirPods Pro as well: a silent head-nod will allow users to answer an incoming call without saying a word out loud to Siri, and contrast, a shake of the head will decline a call. These silent interactions will also be applicable to messages and notifications.

Catch up here for all the news out of Apple’s WWDC 2024.