Thoughts on DALL•E 2 #2

Dall•E 2 understands different art styles. When given this prompt:

An oil painting of a rubber duck wearing medieval robe by Van Gogh

It surprised me with this:

An oil painting of a rubber duck wearing medieval robe by Van Gogh

It even gave these ducks a little toy rubber duck of their own which I thought was a nice touch. I really loved the second image. My only peeve with it was that it was a little too ‘tight’ and cropped the headgear. Fortunately, Dall•E also allows ‘inpainting’. You can erase parts of an image and ask it to fill the missing bits in. I shrunk the original a little and asked Dall•E to generate the rest. The results were more satisfactory:

An oil painting of a rubber duck wearing medieval robe by Van Gogh (‘Inpainting’)

I tested the same prompt but changed the name of the artists.

M.F. Hussain gave these:

An oil painting of a rubber duck wearing medieval robe by M. F. Hussain

and Dali these:

An oil painting of a rubber duck wearing medieval robe by Dali

Vermeer is another of my favourite painters, so naturally my next few prompts asked Dall•E to re-imagine his “A girl with a pearl earring” - but with a raccoon.

“A racoon with a pearl earring” by Johannes Vermeer

produced these:

A racoon with a pearl earring by Johannes Vermeer

I guess “with a pearl earring” doesn’t necessarily say that the pearl earring goes in the ear¹. Only one of the four variants made the connection. I tweaked the prompt and tried to be really explicit about the positioning of the earring but was only partially successful.

A racoon wearing a pearl earring in its ear by Johannes Vermeer

A racoon wearing a pearl earring in its ear by Johannes Vermeer

All that playing with raccoons got me thinking of Rocky Raccoon from the epoynomous The Beatles song. Depiction of violence is strictly forbidden by Dall•E so prompting literally from the lyrics of the song:

Rocky Raccoon
Checked into his room
Only to find Gideon’s Bible
Rocky had come equipped with a gun
To shoot off the legs of his rival

Merely got me a warning about the request not following Dall•E’s content policy.

Dall•E content policy warning

I tried some creative prompt writing to try and coax Dall•E to render the scene from the song:

A raccoon with a book in one hand and a blunderbuss in its other hand by Johannes Vermeer

A raccoon with a book in one hand and a blunderbuss in its other hand by Johannes Vermeer

A raccoon with a book in one hand and a pistol in its other hand by Johannes Vermeer

A raccoon with a book in one hand and a pistol in its other hand by Johannes Vermeer

I got pictures of raccoons with misshapen, incipient gun-like objects that vaguely held capacity for violence, but nothing even close to what Lennon-McCartney intended in their lyrics.

Given the tremendous potential for abuse for generative AI I can see why Open AI (the organisation behind Dall•E) would be extremely cautious about what they allow on their platform. I don’t think this achieves much. All it does is ensure that they don’t have a public relations problem on their hands. Real world depictions of these subjects aren’t going away (is Tarantino going to work on another film?) and now that the generative AI cat is out of the bag, there’ll be other models with no qualms about depiction of violence or religion or some heady mix of other taboo subjects.

October 16, 2022

Thoughts on DALL•E 2 #1

Last week, I tried to move our coffee table (with a few heavy books still on it) to its correct place in the living room. The table is no longer in the wrong place. Unfortunately, now my lower back is. I’ve therefore been at home. A lot.

Fortunately, I also got my DALL•E invite that week which gave me something to fill my time with.

For those of you reading this post who might not have heard about it:

DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.

The Open AI website makes it really simple to use it. You enter your description or “prompt” in a text box, click the generate button, wait for a few seconds and you’ll be presented with four square (1024px by 1024px) images to choose from.

The first prompt I issued to DALL•E was¹

“Astronauts playing cricket on moon”

I got these four pictures back:

Astronauts playing cricket on moon

Three were cartoonish digital art (one had even been titled as if by a first grader - cucket cinn - cricket scene?), while the last one looked sufficiently photo realistic.

Astronauts playing cricket on moon - the photorealistic version

Playing cricket professionally requires you to wear a lot of gear. I liked how DALL•E tried to give the players cricket batting pads under their spacesuits. It also retained the original spacesuit helmet - a cricket helmet alone on moon would be very problematic indeed. The bat in the hands of the player in the front looks something futuristic while the bat in the hands of the player behind is more recognisable. Although its handle is either weirdly long (and broken) or the player is holding a bail in his other hand. The face of the player behind is a distorted smudge. The ball looks like a worn white one - so that settles the question of the game’s format on the moon. There is only a single stump behind the main batsman. There is also a weird piece of debris between the two batsmen - random glitch or is it one of those white discs that fast bowlers use to mark their run up? I guess we’ll never know.

What race or skin tone should the AI have given to these depictions of humans? To say nothing of their gender. Did DALL•E try to cop out of having to settle this question by generating generic illustrations of humans in spacesuits as the first three choices?².

My attempts at getting more specific with this prompt didn’t get me very far:

Astronauts playing cricket on moon with three slips and a gully and earth rising in the background

Produced these:

Astronauts playing cricket on moon with three slips and a gully and earth rising in the background

Again, for the astronaut cricket theme, DALL•E leans towards generating images of low fidelity drawings.

DALL•E makes it really simple to generate variations based on the version you like. Generating variations of the third, somewhat photorealistic image gave results that looked rather disturbing:

Astronauts playing cricket on moon with three slips and a gully and earth rising in the background - photorealistic variations

For some reason the cricket bats seem to take on impossibly long handles - to the point that they become oars. And cricketers themselves seem to take on tortured, contorted forms with nightmarish faces and missing or extra limbs (or parts thereof). In one variant the astro-batter even seems to be attempting to commit Seppuku with their bat.

Staying on with the cricket theme I tried something closer home:

Sachin Tendulkar playing a pull shot while riding an elephant in Mumbai

Which gave:

Sachin Tendulkar playing a pull shot while riding an elephant in Mumbai

Open AI does not allow you to generate faces of celebrities due to the sheer scope of misuse and legal complications that would entail. So no, I wasn’t expecting to see Sachin Tendulkar but some vague likeness of his.

I see Sreesanth in one if I squint really hard but not Sachin.

And I knew from the past experiment that DALL•E has, let’s call it, a serious cricket bat problem, but here it was shockingly inept. Long poles these are, cricket bats these are not. Notice that I did not mention the word “cricket” in my prompt. It still seem to have made that connection somewhat - as evidenced by those white balls flying around and the blue of the Indian cricket jersey.

The elephants look real, even though 6 out of the 8 variations I generated were missing tusks. There is even a reddish stone wall in the background of one of the variants, the sort you are likely to come across at a temple in South India. The AI seem to have made a connection with the likely surrounding of elephants in India (or at least the surrounding that is likely to dominate the pictures in its training set):

Sachin Tendulkar playing a pull shot while riding an elephant in Mumbai - outside a temple perhaps?

The aesthetic of these pictures is definitely redolent of the Subcontinent - there is a small crowd of people in the background in most of them - as there undoubtedly would be were Sachin to venture to do the thing my prompt says. The sunlight is harsh. The colours of clothes is what I’d expect in a random sampling of people from India. And of course, given the setting of our scene, this time the AI seems to have suffered no predicament about the colour of people’s skin. That said, nothing about these images makes me think that are set in Mumbai.

It might be tempting to dismiss DALL•E based on the quality of these results. There is more to DALL•E than generating photorealistic images of cricket in impossibly absurd settings.

I’ll be sharing more examples in the coming days.

August 14, 2022

Westertoren

Some pictures of Westerkerk’s clock tower taken earlier this month:

From the Wikipedia page about Westerkerk:

The tower, called the Westertoren (“Western tower”), is the highest church tower in Amsterdam, at 87 meters (±286 feet). It is not known who the designer of the spire was. Hendrick de Keyser designed an octagonal spire for the tower which was never built. It is suggested Jacob van Campen was the designer. The crown topping the spire is the Imperial Crown of Austria of Maximilian I.[8]

Something I noticed while looking at the pictures closely - they use IIII for the number 4 instead of IV on the clock’s face. Obviously, I am not the first one and this page lists various theories for why this may be.

July 24, 2022

A vignette from a visit to the Acropolis

We were in Athens earlier this year. While the wife and I had traveled individually last year to meet our families, this was our first vacation together after the pandemic.

Athens has a lot to offer and you can easily fill a week and not get bored. The biggest draw for most visitors is the ruins at the Acropolis. You can buy a combi-ticket for 30€ per person that gives you access to 7 key sites and in a way also gives you a ready-made itinerary to follow. You can beat the rush by buying it at one of the (relatively) less popular sites. We procured ours at the temple of the Olympian Zeus.

The temple of Olympian Zeus was the size of a large playground with a raised, rectangular patch in the center where heavily scaffolded Corinthian columns stood on one side and a single column without scaffolding stood next to them. I am a little embarrassed to admit that I realized only just then that Greek columns, especially the 17m+ (55ft+) variety are not monolithic but are usually made up of smaller cylindrical blocks stacked on top of each other.

Wikipedia informs me that:

Construction began in the 6th century BC during the rule of the Athenian tyrants, who envisaged building the greatest temple in the ancient world, but it was not completed until the reign of the Roman Emperor Hadrian in the 2nd century AD, some 638 years after the project had begun.

638 years in the making. Talk about construction delays.

Various fragments of columns and other bits of masonry were scattered about at the perimeter of the site and were cordoned off. Feral cats roamed amidst the ruins - nothing was off-limits to them in Athens.

Right next to the site was The Arch of Hadrian - a Roman triumphal arch. If you stood facing it with your back to the Temple of the Olympian Zeus, you’d see a busy road with unrelenting traffic but if you looked a little further, you’d spot the Acropolis on a hill - our next destination.

On the way to the Acropolis were souvenir shops that were yet to open for the day. This in itself would have been utterly unremarkable had the shutters of these shops not been painted with cartoonish graffiti of ancient Greek myths:

We took a little break at one of the small cafeterias that dot the hilly streets in that area.

I try to think of how the Acropolis entered my consciousness and I am taken back to Yanni’s concert there in 1993. Whether it was broadcast live on national television in India or they showed only clips of it in the news, I don’t remember. But I do remember this long haired mustachioed man dressed in a flowing white shirt and his large orchestra making pleasant (if at that time a tad boring) music at this spectacular location at night. For a middle-class teenager growing up in India of the early 90s, the Acropolis was a distant place that might as well have been on another planet.

The ruins of the Acropolis are imposing even in their dilapidation. One can’t help but wonder how many people (and no doubt animals) it must’ve taken to erect those columns and how long it must’ve taken them without modern machinery and fossil fuels to do it.

The Parthenon, one of the main structures at the Acropolis, had been visible to us from the streets around our hotel. We were delighted to finally see it up close.

The Parthenon was undergoing repairs/restoration - and apparently has been for many years. One side of its facade was covered in scaffolding. With relatively few tourists visiting (given the early time of the year and the assortment of Covid restrictions still in effect), it was quiet here. The shrill buzz of a metal saw cutting stone would occasionally puncture the hush and remind us of the ongoing construction.

Bits of real, reconstructed and restored masonry were again haphazardly strewn all around us. A rather ancient looking guard post opposite the Parthenon had an old dust-covered modem lying inside. I knew the civilizations back then were quite advanced but this looked like a rather significant omission from our history books!

Feral cats were common here too and one of them briefly stopped to sip water pooled in a shallow pit in one of the rocks next to us.

Perhaps the most striking structure at the Acropolis is the Erechtheion, especially the porch of the maidens attached to it. It was cordoned off by rope barriers so we could only look at it from a few meters away but that took nothing away from its grandeur.

A low wall formed a perimeter around the Acropolis. In one corner of the perimeter you could climb a small raised platform and peer over the wall. From here you could see Athen’s dense urban sprawl stretching for miles and hear the distant roar of the traffic. We were even able spot the Arch of Hadrian and the Temple of The Olympian Zeus to our west.

We roamed around a bit longer soaking in the pleasant winter sun and admired other minor structures, amphitheatres and pine trees.

We returned to our hotel exhausted (we had walked 25km - the longest in a single day since the 2020 pandemic started) but with memories that we’ll cherish for many years to come.

p.s. While our ticket allowed entry into Acropolis proper just once, we visited the area around Acropolis several times. A couple of days later we caught a beautiful sunset over the Acropolis from our hotel’s terrace.

April 27, 2022

2021: My year in music

Art is how we decorate space; Music is how we decorate time.

― Jean Michel Basquiat

When I first started the ritual of making an annual playlist, I had no idea if I would be able to sustain it. Turns out, I quite enjoy doing this and look forward to it each year. Here is the sixth iteration with 54 tracks that I loved decorating my time with. With lockdowns of some form or the other throughout 2021 and practically no travel, there certainly was a lot of time to decorate.

Another aspect of this annual ritual used to be writing about tracks in the playlist - both their backstories and their personal significance to me. A bit like liner notes from the days of physical media. The stamina required to do this has definitely been a casualty of the pandemic. Only today, my inability to write these has been further impaired by the fatigue and fever induced by the Covid-19 booster shot the wife and I received just yesterday.

That said, I am more positive than last year about my ability to revisit this playlist throughout 2022 and fill in the stories over the coming days. We’ll see.

Anyway, here is hoping you’ll enjoy this year’s picks:

The 2021 playlist on Spotify

January 1, 2022

The stories of our plants: cosmos bipinnatus

Come spring and the grocery store here starts giving out a small kitchen garden “starter packs” of seeds for every 15€ or so of groceries purchased. Marketed as moestuin maatjes¹ in Dutch, each pack includes a couple of tiny cardboard pots (about 4-5 cm deep), a tablet of soil and different varieties of seeds. All you need to do is fill the pots with moist soil, sow the seeds about 0.5-1cm deep, keep the whole package moist and wait for the seeds to germinate. They even include tiny placards that you can cut out from the packaging and place them in the tiny pots to remind you about what you had planted and when.

Moestuin Maatjes

This annual spring time grocery store ritual must’ve been going on for several years but we never paid much heed to it before. The pandemic revived some atavistic agrarian need in us last year. When the cashiers at the grocery store started offering moestuinje packets around March this year, we took them with both hands.

The wife and I sowed two packs of seed each and waited for them to germinate. In anticipation of the seeds needing a bigger pot after a few weeks, we procured potting soil and clay pebbles. During one of our long weekend walks, we saw a set of empty outdoor planters that someone had left at a bench outside their home for anyone to take for free. Not wanting to cut our walk short or lug them along for our walk, we decided to pick them up on our way back. However, by the time we returned an hour or so later, they were gone. We picked the only one that remained - a small plastic pot with an outer metallic bucket painted bright pink.

A couple of camomile seeds I had sown, germinated some 7-8 days later. Another 5 or so days after that I transferred the two saplings to the planter we had procured on our walk. The growth of the plant stalled shortly thereafter. The cotyledons didn’t give way to true leaves and started withering away. I thought I had over watered the plants to their death. One Sunday morning, as I looked closely at the plant to diagnose what was going wrong, I saw the shoot of the remaining plant move a little. Since this wasn’t beanstalk from the English fairy tale it couldn’t be growing fast enough for me to spot it by merely staring at it for a few seconds. Nor was I on any medication that’d alter the way my mind percieved passage of time. So clearly, another rational interpretation was called for. A few more seconds of close inspection revealed my problem - a larva was gnawing at my plants’ roots and tender shoots.

And so the first batch of seeds came to nought. I let the pot lie fallow for a few days in the balcony while I thought about what I would plant next. We had one moestuin maatje with cosmos seeds lying around and I figured the plant’s beautiful pink flowers might go well with the planter’s outer metal shell and so that’s what I eventually planted.

It’d be a long wait before I’d see the first flower.

Save for a handful of warm days, the summer in Amsterdam this year has been mild. So much so that it has felt like protracted spring. Encouraged by the long days and gentle weather, three of the four cosmos seeds I had planted began to germinate. One of them wilted away after a week but the remaining two started to thrive. Their thread-like² leaves look delicate and like the mother of a child studying in a boarding school worrying about the child eating right, I wondered if they had enough chlorophyll to photosynthesise sufficient food for the plants. The plants also grow quite tall for something this delicate.

Cosmos plant’s linear-filiform leaves

After three months or so of regular care and one round of light application of liquid fertilizer, the first buds began to appear on one of the plants. The stem that seemed to be effortlessly supporting the plant’s 30-40cm height while braving Amsterdam’s windy days, began to buckle a little under the weight of the buds. One of the buds grew to the size of a large blackberry and made it clear that it’d be the first one to blossom. But it gave no sense of what colour the flowers would be.

The anticipation was driving me crazy. A short sampling of my stream of consciousness:

Should I make tiny incision and take a peek inside? No. Would there even be incipient petals that’d give away the colour? No idea. Should’ve paid more attention to botany lectures in school. Should I be throwing the equivalent of a gender reveal party? Not unless I wanted to be locked away in a padded cell.

I did none of the above things and just waited patiently. Each morning, while I wait for the water in the kettle to boil for our tea, I check on the plants in the balcony. A week ago, I was overjoyed to find a large, white cosmos flower where one of the buds had been the day before! While my grand plan to match the colour of the flowers to that of the planter had been scuppered by the flower’s white colour, it was impossible not to run around the house with the pot held like Rafiki raising Simba - for it was the outcome of weeks of worrying, care and waiting (mostly waiting).

White cosmos flowers

The cosmos plant in a pink planter

Also, the second plant is yet to flower. We might still have a match…

Update: Between the time I took me to start writing this and post this, the second plant has begun to blossom. The flowers are pink!

The pink cosmos flower

September 11, 2021