Remove a photo background on your Mac, offline, with Apple's Vision

What you will build

A small command-line tool that removes the background from a photo on your Mac, offline, using Apple's Vision framework. You give it a photo and get back a transparent PNG of the subject, with the same size as the original. A short Python step then cleans the edges and saves a WebP that sits well on both a light and a dark website background. The dainer.ai hero portrait was cut out this way, so it works in both themes.

Before you start

A Mac running macOS 14 (Sonoma) or later. Apple lists VNGenerateForegroundInstanceMaskRequest as available from macOS 14.
Swift on the command line. Check with swift --version. If it is missing, install the Xcode Command Line Tools with xcode-select --install.
Python 3 with Pillow for the edge clean-up: python3 -m pip install pillow.
A photo with a clear subject: a person, a pet or an object in front of a background.
About 10 minutes the first time.

How it works

macOS can lift the main subject out of a photo on the device. It is the same idea as "Copy Subject" in the Photos app. The Vision framework exposes it to code as a request. You run the request on an image, get back an observation that labels each separate subject (Apple calls them instances), and ask the observation to produce a copy of the image where everything except the chosen instances is transparent.

Nothing is uploaded. The photo never leaves your Mac, and there is no account, API key or usage limit.

Read the photo into Core Image, applying its orientation metadata.
Run a VNGenerateForegroundInstanceMaskRequest through a VNImageRequestHandler.
Take the first result, a VNInstanceMaskObservation, and choose which instances to keep.
Call generateMaskedImage to get the subject on a transparent background, at full size.
Write it as a PNG, then clean and compress the edges with Pillow.

Step 1: Check your Mac and Swift

Run these two commands:

BASH

sw_vers -productVersion
swift --version

The first must print 14 or higher. The second must print a Swift version. We ran everything below on macOS 15.6 with Swift 6.1 on Apple Silicon.

Check

Both commands print a version and no error.
If swift asks to install developer tools, accept, then run the command again.

Step 2: Save the script

Save this as cutout.swift. It takes an input path and an output path, and prints the output size and the number of subjects it found.

SWIFT

import Foundation
import Vision
import CoreImage
import AppKit

let args = CommandLine.arguments
guard args.count == 3 else { fatalError("usage: swift cutout.swift <input> <output.png>") }
let input = URL(fileURLWithPath: args[1]), output = URL(fileURLWithPath: args[2])
guard let ci = CIImage(contentsOf: input, options: [.applyOrientationProperty: true]) else { fatalError("cannot read input") }
let handler = VNImageRequestHandler(ciImage: ci, options: [:])
let request = VNGenerateForegroundInstanceMaskRequest()
try handler.perform([request])
guard let result = request.results?.first else { fatalError("no subject found") }
let buffer = try result.generateMaskedImage(ofInstances: result.allInstances, from: handler, croppedToInstancesExtent: false)
let masked = CIImage(cvPixelBuffer: buffer)
let ctx = CIContext()
guard let cg = ctx.createCGImage(masked, from: masked.extent) else { fatalError("render failed") }
let rep = NSBitmapImageRep(cgImage: cg)
try rep.representation(using: .png, properties: [:])!.write(to: output)
print("ok", cg.width, cg.height, result.allInstances.count, "instances")

What each part does:

CIImage(contentsOf:options:) reads the photo. The .applyOrientationProperty option tells Core Image to rotate the image according to its orientation metadata, so a phone photo taken upright does not come out sideways.
VNImageRequestHandler holds the image, and perform runs the request on it.
result.allInstances is an IndexSet of every subject Vision found, not counting the background.
generateMaskedImage(ofInstances:from:croppedToInstancesExtent:) returns a high-resolution image where everything except those instances is transparent. Passing false for cropping keeps the original canvas size, which makes it easy to line the cut-out up with the original photo.

Check

The file is saved as plain text with the .swift extension.
The first line is import Foundation.

Step 3: Run it

BASH

swift cutout.swift portrait.jpg portrait-cut.png

swift compiles and runs the file in one go, so the first run takes a few seconds longer. On our test image, a 1536 by 1024 WebP of a stone sculpture, it printed ok 1536 1024 1 instances and the whole run took under two seconds. About 68% of the pixels came out fully transparent, and a dark-background preview showed a clean outline.

Check

The output PNG opens in Preview with a checkerboard where the background was.
The printed width and height match the original photo.
The instance count makes sense: one person, one instance.

Step 4: Keep only the subjects you want

When a photo has more than one subject, for example two people, Vision labels each as a separate instance. The labels start at 1, because the background is not an instance. To see them, print Array(result.allInstances). To keep only the first subject, replace result.allInstances in the generateMaskedImage call with:

SWIFT

IndexSet(integer: 1)

We checked this on our single-subject test: allInstances printed [1], and the mask from IndexSet(integer: 1) was identical to the mask from allInstances.

If you want the mask itself, for example to feather it in another tool, VNInstanceMaskObservation also has generateScaledMaskForImage(forInstances:from:), which returns a full-resolution mask instead of the masked photo.

Check

With two people in the frame, keeping instance 1 and then instance 2 gives two different cut-outs.
The cut-out still has the original canvas size.

Step 5: Clean the edges and save a web file

The mask can leave a faint halo along the edge of the subject, and on our portrait it showed against the light background. Shrinking the alpha channel by one pixel removes the halo, and a very small blur softens the new edge. Then save as WebP for the web.

PYTHON

from PIL import Image, ImageFilter
im = Image.open("portrait-cut.png").convert("RGBA")
alpha = im.getchannel("A").filter(ImageFilter.MinFilter(3)).filter(ImageFilter.GaussianBlur(0.6))
im.putalpha(alpha)
im.save("portrait-cut.webp", "WEBP", quality=90)

MinFilter(3) picks the lowest value in each 3 by 3 window, which pulls the edge of the alpha in by one pixel. GaussianBlur(0.6) then softens that edge slightly. Both are standard Pillow filters. Keep the blur small so the edge stays crisp; raise it only if edges look jagged.

Check

The WebP keeps transparency: open it in a browser on a dark page and the background shows through.
The file is much smaller than the PNG. Our 1.0 MB test PNG became a 116 KB WebP at quality 90.

Step 6: Preview on light and dark backgrounds

Before you put it on the site, look at it on both of your theme colours. This short script writes two previews using your page background colours.

PYTHON

from PIL import Image
cut = Image.open("portrait-cut.webp").convert("RGBA")
for name, colour in [("light", (246, 244, 238, 255)), ("dark", (20, 22, 19, 255))]:
    bg = Image.new("RGBA", cut.size, colour)
    Image.alpha_composite(bg, cut).convert("RGB").save(f"preview-{name}.png")

Replace the two colours with your site's light and dark backgrounds.

Check

No light rim around the subject on the dark preview.
No dark fringe on the light preview.
Fine details, like glasses frames or loose hair, look acceptable at the size you will show them.

Step 7: Batch a folder of photos

The script is a normal command, so a shell loop handles a folder:

BASH

mkdir -p cut
for f in photos/*.jpg; do
  swift cutout.swift "$f" "cut/$(basename "${f%.*}").png" || echo "skipped $f"
done

The || echo keeps the loop going when one photo has no clear subject.

Check

One PNG per photo in cut/, and a skipped line for any photo that failed.
Spot-check a few results on the dark preview before using them.

Step 8: Use it on your site

Serve the WebP with explicit width and height attributes on the <img>, so the page does not jump while it loads. Because the background is transparent, the same file works on a light and a dark theme. On dainer.ai the cut-out portrait sits in the hero on either theme.

Check

Capture the page in light and dark mode at phone and desktop widths and look at the portrait edges at real size.

Gotchas we hit

A portrait with a background only fits one theme. The original hero photo carried its own background, which suited one theme and not the other. Cutting out the subject made one file work on both.
A faint halo on the light background. The raw mask left a thin rim along the edge. The one-pixel alpha shrink plus a 0.6 px blur removed it.
Online removers were not an option. They need the photo on someone else's server. Vision does the same job on the Mac with no upload.

Two guards in the script come from how the APIs behave rather than from a failure we saw: it stops with no subject found instead of writing an empty file when Vision returns no observation, and it applies the photo's orientation flag so an upright phone photo is not processed sideways.

Take it further

Convert HEIC or other formats first with the built-in sips tool if Core Image cannot read a file.
Wrap the Swift script in a Quick Action or Shortcut so you can right-click a photo in Finder.
Add a drop shadow in the layout with CSS rather than baking it into the image, so it can change per theme.
Read AI-assisted visual QA to capture the finished page in both themes.

Quick checklist

macOS 14 or later, and swift --version works.
The script reads with .applyOrientationProperty and keeps the original canvas size.
You chose which instances to keep, or kept them all on purpose.
Edges were shrunk by one pixel and softened slightly.
The result was previewed on both theme backgrounds.
The web file is WebP with transparency and explicit dimensions in the page.
No photo left your Mac.