Why I Suspect there is an Absence of Photorealism in Apple's AI-Generated Images

I've read some commentary here and there about how the images Apple Intelligence generates are insufficiently photorealistic. One member of the MacStories' Discord (apologies for the lack of credit where credit’s due — there’s been a lot of discussion there) suggested that the image quality, in this regard, might improve in time as Apple’s models ingest more and develop further.

I observed there that I suspected the choices Apple made about the kinds of images generated by its Machine Learning might be intentional. With machine learning tools that remove people and objects in the background coming to Photos, Apple is showing its models are capable of photorealistic renders.

So why might they not permit Image Playground to do the same?

I can see where some of this may be limitations dictated by on-device computational power. While I’m no engineer,  I would guess fixing/repairing a of a photograph seems like it would be asking less of a Neural Engine than creating a full photo. Even so, the use of cloud computing would be an easy enough way to get around this.

Rather, I suspect it has everything to do with user responses to the generated images and, sadly, the motivations some users might have. Transforming a prompt into an image that is obviously not real is usually more than enough to meet most users' needs. The creation of a gazebo for a Keynote, to use Apple's example, is one where a visual concept is being communicated and that communication does not suffer from a lack of photorealism.

But there are cases where people can and will suffer from the malicious deployment of photorealistic renders. Indeed, in a rare case of high profile bipartisanship, The US Congress is (Finally!) moving to criminalize deepfake revenge porn. Senators Ted Cruze (R) is working with Amy Klobuchar (D), who both were candidates for their party's nomination for the presidency, find easy common ground here.

As well they should. it has been reported that middle schoolers in Florida and California (and, I suspect, elsewhere) — a demographic seldom used in profiles of good sense and sound decision-making —have learned they can use AI to generate photorealistic nudes of their classmates.

There's a reason we find The Lord of the Flies plausible — even if the historical event turned out for better than fiction.

It's the kind of problem that even a n AI enthusiastic techbro focused on making the Star Trek computer or their own personal Jarvis should have seen coming because it always seems to happen.

It was obvious enough to come up in a meeting at Apple.

And sidestepping photorealism is an obvious solution.

Keeping their Image Playground in check in this regard makes good sense and is, I would argue, protecting its users from those who might use the its technology for malicious purposes.

Confused by the Confusion (or, What iPad Should I Buy?)

With the release of the newest version of the Apple Pencil, I have begun to see a number of pundits and journalists talk about how confusing the iPad line has become. It is tempting to respond to this with a condescending claim about their own condescension towards "average" consumers — even if it would be wrong to do so.

Those poor, benighted non-techies really can't handle anything beyond the good/better/best progression ninety percent of the time so Apple needs to get rid of those extra models in the lineup, they might say.

Well, I might respond with equal condescension, you just don't get the iPad because your frame of reference is bound to the fourfold division of the Mac promulgated by Steve Jobs when he returned from NEXT, with desktop/ laptop on one axis and consumer/pro on the other — a model that doesn't fit the iPad. (Even though the model kind of does fit...)

Other than providing the smug satisfaction that comes along with any good strawman logical fallacy, this does no one any good. The pundits and reviewers claiming confusion and citing the clarity that adding a larger iPad Air would bring are expressing a real problem that they face every time a friend or family member asks what device they should buy.

It's one thing to offer the wrong advice to a reader or viewer you will never meet. It's another to get to hear the griping every holiday dinner and suffer through the implication that you are bad at your job.

It is equally true that the new Apple Pencil appears to be hinting that there is some kind of change coming. Whether it is a hardware redesign (Does USB-C signal a move away from magnetic charging to make space for a repositioned FaceTime camera?) or a shift in design philosophy (We need a good/better/best progression for the Apple Pencil.) remains to be seen.

That said, the decision tree as to which iPad to buy is simple.

1. Is price the most important thing for you? If yes, get the basic iPad. If not, go to question two.

2. Are you a videographer or professional (or similarly serious) photographer? If yes, get the Pro. If not, go to question 3.

3. Are you looking for maximum portability? Get the Mini.

4. If not, get the Air.

Yes, at each of these steps, there are other questions but those answers come easily. How much memory should I get? How much are you using? What about the screen size? Those who need a big screen or just love the Mini know who they are already.

Those are questions that can be walked through simply enough and most consumers can figure those out.

That said, I want to go back to that two-by-two grid for a second, because it is hiding in the iPad lineup. If you look at those four questions, and don't get distracted by form factor, you can set up the grid -- based on most likely place of use -- with laptop replacement/desktop replacement on one axis and consumer/ professional on the other. The consumer laptop is the Mini. The consumer desktop is the Air ( living in its Magic Keyboard case). The pro laptop is the 11 inch Pro and its desktop counterpart is the 12.9 inch.

The more peripherals you add (e.g., keyboards, track pads, and external monitors), the clearer the alignment becomes.

If Apple were to add a larger iPad Air to the lineup, the grid gets simpler and the Mini becomes more of a specialty item, much as the base iPad is targeted at the education market. Like the Pro, you will know if you need it — either because it speaks to you or because you want a un companion for your Mac.

I am not going to argue iPads and Macs have become interchangeable. They aren't and each comes with strengths and limitations. What I will argue is that there is more consistency and less confusion than may meet the eye for the average consumer than we techies might initially think.

Managing Stages with Apple

Apple takes a much longer view with its hardware and software than most people realize. Developers are annually reminded of this at Apple's Worldwide Developer’s Conference (WWDC) when they are gently prodded to update their apps to use newer APIs -- sometimes years in advance of the hardware changes the APIs are designed to support..

This long term approach exists in tension with Apple's secretive approach to its future releases. As a result, so much of the innovative work being done only becomes clear in hindsight.

Take, for example, CarPlay. The new dashboard display that came with iOS13 was a fresh, new look—something more useful than the grid of apps. And having a map and some tappable controls available was a nice, creature comfort advance.

After the release of widgets in iOS and iPadOS the following year, however, the design of CarPlay began to look like a precursor of things to come. Now that widgets have became increasingly interactive, it is now obvious that Apple had been playing with the concept of a simplified, tappable interface for iPhones and iPads prior to its open deployment for users.

It's anyone's guess if this was always Apple's plan or if, one day, a clever engineer looked up, turned to their colleagues, and said, "You know, we could do this on a phone, too!”

I was considering this as I listened to the tvOS review on App Stories this morning — especially the parts about where Apple could go with its iOS equivalent of the Mac Mini.

Let me add to the wish list for such a device.

There is an underused USB part on the back of every AppleTV. Those USB ports are obvious places to connect a media server or a backup drive. That puts us one app away from an iOS-based Time Machine app.

As my old Mac Mini limps toward retirement, I would like to have a new method of backing up our family photos.in an ideal world, I would prefer an ids-based solution. A more advanced Apple TV is an obvious solution — especially for a growing customer base that is less interested in maintaining a macOS infrastructure alongside their iOS/iPadOS infrastructure.

In saying this, I recognize this may sound like a concern from a different era. But there is something about your child's baby pictures that makes you want to take the belt and suspenders approach to backing up your data.

The Apple Product No One is Talking About (and Why Educators Should Care)

In the space between iOS11 and MacOS High Sierra, I wanted to offer a few words not he big product Apple announced that (with some exceptions) has gotten little to no air time. 

I'm talking about the Apple Town Squares, formerly known as the Apple Stores.

It's worth listening to what Tim Cook and Angela Ahrendts talk about, in terms of the relationship of space and purpose, beginning four minutes and four seconds into the Apple Keynote of 12 September 2017. They spoke of the design philosophy behind Apple Park and the Apple Town Squares. Cook spoke about how Steve Jobs set out to "inspire talented people to do their best work" and that the retail spaces were designed to be "about learning, inspiring, and connecting with people" as much as it was about retail. Ahrendts spoke about the redesign of the flagship stores to serve as "gathering places for 500 million people" and that they are "Apple's largest product".

If you listen to the way they spoke about Apple's facilities, they can easily describe some of the central aspects of a University. In case that is less than immediately apparent, here are some of the parallels between Apple's facilities and a university's -- if the clearly intentional parallel was not driven home enough when Ahrendts linked the Creative Pros to the Liberal Arts and the Genius to Technology (STEM, for those attached to the current nomenclature of the university):

The Plaza = The Quad

The Forum = The Classroom

The Board Room = Library Study Spaces

The Genius Grove = Faculty Office Hours

The Avenues = The Student Union, with its opportunities for students

Her talk highlighted Apple's efforts to offer Lifelong Learning opportunities -- for free -- to its community. Those who are in the business of supplying such opportunities to potential students at local universities should both take note, as Apple appears to be doing this in a more compelling and targeted manner than universities are, and be alarmed, as Apple is beginning to eat the lifelong learning lunch.

There is more, however. Apple Park and Apple's Town Squares are described in a language that, as I suggested, parallel the university. Given that universities are trying to figure out how to bring students to their campuses in an age when, quite frankly, there aren't enough students to go around and online learning and similar innovations might keep potential students away, Apple's successful strategies should be examined closely by those teams trying to figure out how to differentiate their campus in a way that makes it a compelling destination.

---

Dr. Matthew M. DeForrest is a Professor of English and the Mott University Professor at Johnson C. Smith University. The observations and opinions he expresses here are his own. You are very welcome to follow him on Twitter and can find his academic profile at Academia.edu.