I shipped my first visionOS app about six months ago. It took three times longer than expected, I rewrote the spatial UI twice, and I almost rage-quit when RealityKit decided to ignore my collision shapes for the fourth time. But I learned a ton, and now I’m going to dump everything I wish I’d known on day one.

What You Need Before Starting

Hardware:

A Mac with Apple Silicon (M1 or later). Intel Macs can’t run the visionOS simulator. Non-negotiable.
Ideally an M2 Pro or better — the simulator is a memory hog and an M1 MacBook Air will struggle with complex scenes
An actual Vision Pro for testing is… look, it’s technically optional. The simulator works for 80% of development. But the other 20% — spatial interactions, eye tracking, hand gestures — you really need the device. If you don’t have $3,500 for a dev kit, find a friend or hit up an Apple Developer Center.

Software:

Xcode 16+ (latest version, always)
visionOS SDK
Reality Composer Pro (included with Xcode)
A developer account ($99/year)

Knowledge prerequisites:

Swift and SwiftUI. If you’re coming from React or Unity, SwiftUI will feel weird for about two weeks and then you’ll love it. Or at least tolerate it.
Basic understanding of 3D coordinate systems. You don’t need to be a game developer, but knowing what a transform matrix does helps.

The Three App Types (Pick One)

Apple gives you three paradigms for visionOS apps. This is the first decision you’ll make and it’s important.

Window Apps

Regular 2D windows that float in space. Think iPad apps but they hover in your living room. If you’re porting an existing iOS app, start here. SwiftUI runs basically the same way, just with some spatial considerations.

Volume Apps

3D content contained in a bounded volume — like a snow globe of your app. The user can walk around it, but the content stays within its box. Good for data visualization, 3D model viewers, product demos.

Full Space Apps

Your app takes over the entire spatial environment. This is where immersive VR/AR experiences live. Games, virtual travel, simulations. Most powerful, most complex.

My advice: Start with a Window app. Even if your goal is a Full Space experience, getting comfortable with the visionOS development workflow in a simpler context will save you time. I jumped straight into Full Space and spent two weeks fighting the framework instead of building my app.

The Frameworks, Simplified

SwiftUI — Your UI framework. Buttons, text, lists, navigation. Same as iOS, mostly.

RealityKit — Your 3D rendering engine. Handles models, animations, physics, spatial audio. This is where most of your time goes if you’re building anything 3D.

ARKit — Access to Vision Pro’s sensors. Hand tracking, scene understanding, plane detection, world anchoring. You need entitlements for some of these features.

Reality Composer Pro — Visual tool for building 3D scenes. Think of it as Interface Builder but for spatial content. It’s actually pretty good for prototyping but I find myself doing most work in code eventually.

Common Pitfalls I Hit

1. The Coordinate System Will Trip You Up

visionOS uses a right-handed coordinate system where Y is up and Z comes toward you. If you’re coming from Unity (left-handed, Y up) or web 3D (often Z up), your models will be rotated wrong and you’ll spend an hour figuring out why your floor is a wall.

2. Collision Shapes Need to Be Simple

RealityKit’s collision detection works great with primitive shapes — boxes, spheres, capsules. Complex mesh colliders are technically supported but they’re buggy and expensive. I spent three days debugging gesture interactions that weren’t firing because my collision shape was too complex and the system was silently ignoring it.

Just use simple shapes. Seriously.

3. Eye Tracking Privacy Is Serious

You don’t get raw eye tracking data. Period. Apple gives you indirect hover effects and gaze-based interaction through the system, but you can’t know exactly where the user is looking. This is a privacy feature, not a limitation — but it means any eye-tracking-dependent interaction needs to go through Apple’s input system.

4. The Simulator Lies

The simulator is useful but it doesn’t replicate the real experience. Gestures feel different on actual hardware. Performance characteristics are different. Spatial audio doesn’t work the same way. And the whole concept of “spatial presence” — how it feels to have virtual objects in your real space — simply can’t be simulated on a flat screen.

Test on device as early and as often as possible.

5. 3D Asset Pipeline Is a Pain

Getting 3D models into your app is… not great. Apple wants USDZ format. Most 3D artists work in FBX, OBJ, or glTF. Reality Converter handles some conversions, but I’ve hit issues with materials, animations, and scale more times than I can count.

My workflow: model in Blender, export as glTF, convert in Reality Converter, import into Reality Composer Pro, fix whatever broke. Every time.

A Minimal First Project

If you want to get something running in under an hour, try this:

Open Xcode, create a new visionOS app (Volume type)
In your ContentView, add a RealityView that displays a simple 3D sphere
Add a tap gesture that changes the sphere’s color
Run in the simulator

That’s your hello world. From there, try:

Loading a USDZ model instead of a primitive
Adding spatial audio to the model
Making the model respond to hand gestures
Adding a 2D SwiftUI panel next to the 3D content

Each of these steps teaches you something about how the frameworks connect.

Resources That Actually Help

Apple’s visionOS documentation — Good but dense. The “Creating your first visionOS app” tutorial is worth doing start to finish.
Apple’s sample code — Better than the docs, honestly. Download the Hello World and Destination Video samples and study them.
WWDC videos — The 2023 and 2024 sessions on visionOS are excellent. Watch them at 1.5x.
r/visionosdev — Small but active community. Good for specific troubleshooting.
Reality School on YouTube — Best video tutorials I’ve found for RealityKit and Reality Composer Pro.

Skip the Udemy courses. Most of them are outdated within months because the SDK changes fast.

The Business Reality

Honest talk. The Vision Pro install base is small. Like, really small. You’re not going to get rich selling a $4.99 app to a few hundred thousand users. The people making money on visionOS right now are either enterprise developers (custom business apps at contract rates) or Apple Design Award-type apps that Apple features prominently.

But — and this is why I’m still building for it — the platform is growing, the developer tools are maturing, and early developers have a real advantage. When the rumored lower-cost Apple headset launches, the install base will jump dramatically, and developers who already understand the platform will be years ahead.

That’s the bet. It’s a long-term play. If you need revenue this quarter, build for Quest instead.

Getting Your App Reviewed

Apple’s visionOS app review is strict. A few things that’ll get you rejected:

Requiring specific room sizes or configurations
Not supporting both seated and standing use
Performance below 90fps
Privacy violations with spatial data
Missing accessibility features

Test against the Human Interface Guidelines before submitting. I got rejected twice before I learned to read them carefully.