Text-to-image AI: powerful, easy-to-use technology for making art – and fakes

A synthetic image generated by mimicking real faces, left, and a synthetic face generated from the text prompt ‘a photo of a 50-year-old man with short black hair,’ right. Hany Farid using StyleGAN2 (left) and DALL-E (right), CC BY-ND

Hany Farid, University of California, Berkeley

Type “Teddy bears working on new AI research on the moon in the 1980s” into any of the recently released text-to-image artificial intelligence image generators, and after just a few seconds the sophisticated software will produce an eerily pertinent image.

Seemingly bound by only your imagination, this latest trend in synthetic media has delighted many, inspired others and struck fear in some.

Google, research firm OpenAI and AI vendor Stability AI have each developed a text-to-image image generator powerful enough that some observers are questioning whether in the future people will be able to trust the photographic record.

an image of three tiny bears standing on the sandy soil in front of an electronic device
This image was generated from the text prompt ‘Teddy bears working on new AI research on the moon in the 1980s.’ Hany Farid using DALL-E, CC BY-ND

As a computer scientist who specializes in image forensics, I have been thinking a lot about this technology: what it is capable of, how each of the tools have been rolled out to the public, and what lessons can be learned as this technology continues its ballistic trajectory.

Adversarial approach

Although their digital precursor dates back to 1997, the first synthetic images splashed onto the scene just five years ago. In their original incarnation, so-called generative adversarial networks (GANs) were the most common technique for synthesizing images of people, cats, landscapes and anything else.

A GAN consists of two main parts: generator and discriminator. Each is a type of large neural network, which is a set of interconnected processors roughly analogous to neurons.

Tasked with synthesizing an image of a person, the generator starts with a random assortment of pixels and passes this image to the discriminator, which determines if it can distinguish the generated image from real faces. If it can, the discriminator provides feedback to the generator, which modifies some pixels and tries again. These two systems are pitted against each other in an adversarial loop. Eventually the discriminator is incapable of distinguishing the generated image from real images.


Just as people were starting to grapple with the consequences of GAN-generated deepfakes – including videos that show someone doing or saying something they didn’t – a new player emerged on the scene: text-to-image deepfakes.

In this latest incarnation, a model is trained on a massive set of images, each captioned with a short text description. The model progressively corrupts each image until only visual noise remains, and then trains a neural network to reverse this corruption. Repeating this process hundreds of millions of times, the model learns how to convert pure noise into a coherent image from any caption.

a house cat with bulky opaque goggles on its face
This photolike image was generated using Stable Diffusion with the prompt ‘cat wearing VR goggles.’ Screen capture by The Conversation, CC BY-ND

While GANs are only capable of creating an image of a general category, text-to-image synthesis engines are more powerful. They are capable of creating nearly any image, including images that include an interplay between people and objects with specific and complex interactions, for instance “The president of the United States burning classified documents while sitting around a bonfire on the beach during sunset.”

OpenAI’s text-to-image image generator, DALL-E, took the internet by storm when it was unveiled on Jan. 5, 2021. A beta version of the tool was made available to 1 million users on July 20, 2022. Users around the world have found seemingly endless ways to prompt DALL-E, yielding delightful, bizarre and fantastical imagery.

A wide range of people, from computer scientists to legal scholars and regulators, however, have pondered the potential misuses of the technology. Deep fakes have already been used to create nonconsensual pornography, commit small- and large-scale fraud, and fuel disinformation campaigns. These even more powerful image generators could add jet fuel to these misuses.

Three image generators, three different approaches

Aware of the potential abuses, Google declined to release its text-to-image technology. OpenAI took a more open, and yet still cautious, approach when it initially released its technology to only a few thousand users (myself included). They also placed guardrails on allowable text prompts, including no nudity, hate, violence or identifiable persons. Over time, OpenAI has expanded access, lowered some guardrails and added more features, including the ability to semantically modify and edit real photographs.

Stability AI took yet a different approach, opting for a full release of their Stable Diffusion with no guardrails on what can be synthesized. In response to concerns of potential abuse, the company’s founder, Emad Mostaque, said “Ultimately, it’s peoples’ responsibility as to whether they are ethical, moral and legal in how they operate this technology.”

Nevertheless, the second version of Stable Diffusion removed the ability to render images of NSFW content and children because some users had created child abuse images. In responding to calls of censorship, Mostaque pointed out that because Stable Diffusion is open source, users are free to add these features back at their discretion.

The genie is out of the bottle

Regardless of what you think of Google’s or OpenAI’s approach, Stability AI made their decisions largely irrelevant. Shortly after Stability AI’s open-source announcement, OpenAI lowered their guardrails on generating images of recognizable people. When it comes to this type of shared technology, society is at the mercy of the lowest common denominator – in this case, Stability AI. Text-to-image generators could make it easier for people to create deepfakes.

Stability AI boasts that its open approach wrestles powerful AI technology away from the few, placing it in the hands of the many. I suspect that few would be so quick to celebrate an infectious disease researcher publishing the formula for a deadly airborne virus created from kitchen ingredients, while arguing that this information should be widely available. Image synthesis does not, of course, pose the same direct threat, but the continued erosion of trust has serious consequences ranging from people’s confidence in election outcomes to how society responds to a global pandemic and climate change.

Moving forward, I believe that technologists will need to consider both the upsides and downsides of their technologies and build mitigation strategies before predictable harms occur. I and other researchers will have to continue to develop forensic techniques to distinguish real images from fakes. Regulators are going to have to start taking more seriously how these technologies are being weaponized against individuals, societies and democracies.

And everyone is going to have to learn how to become more discerning and critical about how they consume information online.

This article has been updated to correct the name of the company Stability AI, which was misidentified.

Hany Farid, Professor of Computer Science, University of California, Berkeley

This article is republished from The Conversation under a Creative Commons license. Read the original article.


What social media regulation could look like: Think of pipelines, not utilities

Is the law coming for Twitter, Meta and other social media outlets? new look casting/iStock via Getty Images

Theodore J. Kury, University of Florida

Elon Musk’s takeover of Twitter, and his controversial statements and decisions as its owner, have fueled a new wave of calls for regulating social media companies. Elected officials and policy scholars have argued for years that companies like Twitter and Facebook – now Meta – have immense power over public discussions and can use that power to elevate some views and suppress others. Critics also accuse the companies of failing to protect users’ personal data and downplaying harmful impacts of using social media.

As an economist who studies the regulation of utilities such as electricity, gas and water, I wonder what that regulation would look like. There are many regulatory models in use around the world, but few seem to fit the realities of social media. However, observing how these models work can provide valuable insights. Families across the U.S. are suing social media companies over policies that they argue affected their children’s mental health.

Not really economic regulation

The central ideas behind economic regulation – safe, reliable service at fair and reasonable rates – have been around for centuries. The U.S. has a rich history of regulation since the turn of the 20th century.

The first federal economic regulator in the U.S. was the Interstate Commerce Commission, which was created by the Interstate Commerce Act of 1887. This law required railroads, which were growing dramatically and becoming a highly influential industry, to operate safely and fairly and to charge reasonable rates for service.

The Interstate Commerce Act reflected concerns that railroads – which were monopolies in the regions that they served and provided an essential service – could behave in any manner they chose and charge any price they wanted. This power threatened people who relied on rail service, such as farmers sending crops to market. Other industries, such as bus transportation and trucking, would later be subjected to similar regulation.

Individual social media companies don’t really fit this traditional mold of economic regulation. They are not monopolies, as we can see from people leaving Twitter and jumping to alternatives like Mastodon and Post.

While internet access is fast becoming an essential service in the information age, it’s debatable whether social media platforms provide essential services. And companies like Facebook and Twitter don’t directly charge people to use their platforms. So the traditional focus of economic regulation – fear of exorbitant rates – doesn’t apply.

Fairness and safety

In my view, a more relevant regulatory model for social media might be the way in which the U.S. regulates electricity grid and pipeline operations. These industries fall under the jurisdiction of the Federal Energy Regulatory Commission and state utility regulators. Like these networks, social media carries a commodity – here it’s information, instead of electricity, oil or gas – and the public’s primary concern is that companies like Meta and Twitter should do it safely and fairly.

In this context, regulation means establishing standards for safety and equity. If a company violates those standards, it faces fines. It sounds simple, but the practice is far more complicated.

First, establishing these standards requires a careful definition of the regulated company’s roles and responsibilities. For example, your local electric utility is responsible for delivering power safely to your home. Since social media companies continuously adapt to the needs and wants of their users, establishing these roles and responsibilities could prove challenging.

Texas attempted to do this in 2021 with HB 20, a law that barred social media companies from banning users based on their political views. Social media trade groups sued, arguing that the measure infringed upon their members’ First Amendment rights. A federal appellate court blocked the law, and the case is likely headed to the Supreme Court.

A woman in a suit testifies before a  congressional committee.
President Biden named Lina Khan, a prominent critic of Big Tech companies, as chair of the Federal Trade Commission in 2021. The agency investigates issues including antitrust violations, deceptive trade practices and data privacy lapses. AP Photo/Saul Loeb

Setting appropriate levels of fines is also complicated. Theoretically, regulators should try to set a fine commensurate with the damage to society from the infraction. From a practical standpoint, however, regulators treat fines as a deterrent. If the regulator never has to assess the fine, it means that companies are adhering to the established standards for safety and equity.

But laws often inhibit agencies from energetically policing target industries. For example, the Office of Enforcement at the Federal Energy Regulatory Commission is concerned with safety and security of U.S. energy markets. But under a 2005 law, the office can’t levy civil penalties higher than US$1 million per day. In comparison, the cost to customers of the California power crisis of 2000-2001, fueled partially by energy market manipulation, has been estimated at approximately $40 billion.

In 2022 the Office of Enforcement settled eight investigations of violations that occurred from 2017 to 2021 and levied a total of $55.5 million in penalties. In addition, it opened 21 new investigations. Clearly, the prospect of a fine from the regulator is not a sufficient deterrent in every instance.

From legislation to regulation

Congress writes the laws that create regulatory agencies and guide their actions, so that’s where any moves to regulate social media companies will start. Since these companies are controlled by some of the wealthiest people in the U.S., it’s likely that a law regulating social media would face legal challenges, potentially all the way to the Supreme Court. And the current Supreme Court has a strong pro-business record.

If a new law withstands legal challenges, a regulatory agency such as the Federal Communications Commission or the Federal Trade Commission, or perhaps a newly created agency, would have to write regulations establishing social media companies’ roles and responsibilities. In doing so, regulators would need to be mindful that changes in social preferences and tastes could render these roles moot.

Finally, the agency would have to create enforcement mechanisms, such as fines or other penalties. This would involve determining what kinds of actions are likely to deter social media companies from behaving in ways deemed harmful under the law.

In the time it would take to set up such a system, we can assume that social media companies would evolve quickly, so regulators would likely be assessing a moving target. As I see it, even if bipartisan support develops for regulating social media, it will be easier said than done.

Theodore J. Kury, Director of Energy Studies, University of Florida

This article is republished from The Conversation under a Creative Commons license. Read the original article.


Why is astronomy a science but astrology is not?

Your zodiac sign – like Sagittarius, the archer – might be in the stars, but your future isn’t. scaliger/iStock via Getty Images Plus

Talia Dan-Cohen, Arts & Sciences at Washington University in St. Louis and Carl Craver, Arts & Sciences at Washington University in St. Louis

Curious Kids is a series for children of all ages. If you have a question you’d like an expert to answer, send it to

Why is astronomy a science, but not astrology? – Katelyn, age 11, Arlington, Texas

Are you sure astrology isn’t a science?

Both astrology and astronomy are in the business of making predictions. The theories of astrology claim that the positions of the planets and the stars influence who you are and what happens to you: your job, your personality and your romantic partner. Astrologers make these predictions based on the positions of the planets at the time of your birth.

Astronomy, in contrast, makes predictions about such phenomena as the movements of planets and the expansion of galaxies. Astronomers explain their predictions with such properties as masses, distances and gravitational forces.

As a philosopher and an anthropologist who study what science means to society, we think it is important to separate the question of whether something is a science from the question of whether it is true or false.

Astrology makes scientific claims

Science, in essence, involves making and testing factual claims about the world. Factual claims are true or false descriptions of the world (Joe is 1 meter tall) as opposed to descriptions of how we define things (1 meter is 1,000 milimeters). In this sense, astrologers, like astronomers, make factual claims about the world. To us, that makes astrology sound a lot like a set of scientific beliefs.

For a very long time, until the 17th or 18th century, astronomy and astrology were practiced side by side. After all, knowing where the planets were relative to the stars was necessary to make accurate predictions about how their locations influenced human affairs. That’s why astronomers and astrologers populated medical schools and governments, advising people on what the heavens signaled was to come on Earth.

Even famed astronomers Galileo and Kepler practiced astrology. Any rule that says they are scientists only if they make one set of factual claims but not when they make another set of factual claims divides these thinkers into two halves that aren’t meant to be contradictory. In both cases, they wanted to know how things worked so they could predict how things would go in the future. For centuries, astrology was a respected science right alongside astronomy.

Being false vs. being unscientific

But here’s the rub: When researchers test the predictions astrology makes about people’s lives, those predictions turn out to be no better than guesswork.

There is currently no broadly accepted evidence that galactic forces are capable of influencing the choices people make. The truck parked on the street exerts more gravitational pull on you than Mars does, and the radio waves from your local station far outpower those from Jupiter, for instance.

There is an important difference between being false and being unscientific. Currently, astrological theories are false precisely because they make scientific claims about the world, and those claims turn out to be wrong. Although the predictions astrology makes are false, they are nonetheless a matter of science. That’s how we know they are wrong, after all.

Diagram of constellations
Image from ‘Astronomy Without a Telescope’ (1869). Internet Archive Book Images/Flickr

Some people believe they find support for astrological predictions in their own personal experience. They read their horoscope and it seems just right: They did “meet someone interesting” or “benefit from listening to a close friend’s advice.” But the predictions are vague enough that they would often be true even if astrology were utterly bogus. That’s why it can be difficult to figure out how to assess an astrologer’s predictions with precision.

Theories of astronomy, on the other hand, have evolved over the years with advances in technology. They are routinely corrected in response to increasingly precise measurements. For example, Einstein’s theory of general relativity got a boost over Newton’s because it predicted the precise migration of Mercury’s closest point to the Sun year after year. If astrology had the same ability to make correct predictions with such precision, it might still be a major focus of scientific attention.

Why is astrology still popular?

But then why do so many people find astrology so useful if its predictions are not well founded? Why are astrological signs and horoscopes so popular?

It seems that looking to the sky to make some sense of what’s going on right now and what’s going to happen in the future has appealed to a lot of different people at different times in history all over the world.

When it comes to what’s commonly known as Western astrology, many people find their astrological sign to be a source of meaning in their lives. In fact, nearly 30% of Americans believe in astrology. It’s one of many tools we have for telling stories about ourselves to make sense of who we are, why we are that way and why experiences that otherwise would feel meaningless and confusing seem to happen to us all the time. In this sense, astrology’s success might be less about prediction and more about what it offers in terms of meaning and interpretation.

Silhouette of person looking up at a night sky next to camera.
Throughout history, people have looked to the stars to derive some form of meaning from existence. Christianto Soning/EyeEm via Getty Images

Among other things, astrology can be a useful prompt for self-reflection. It asks us whether we have traits typical of our astrological sign, and whether those we love have traits the theory suggests they ought to have. Thinking about our traits and relationships with the people around us is generally a good tool for understanding who we are, what we want to be and the meaning of our lives. Perhaps astrology is helpful in this way, independently of whether those traits are fixed by the stars.

Hello, curious kids! Do you have a question you’d like an expert to answer? Ask an adult to send your question to Please tell us your name, age and the city where you live.

And since curiosity has no age limit – adults, let us know what you’re wondering, too. We won’t be able to answer every question, but we will do our best.

Talia Dan-Cohen, Associate Professor of Sociocultural Anthropology, Arts & Sciences at Washington University in St. Louis and Carl Craver, Professor of Philosophy and Philosophy-Neuroscience-Psychology, Arts & Sciences at Washington University in St. Louis

This article is republished from The Conversation under a Creative Commons license. Read the original article.


Heart rate variability – what to know about this biometric most fitness trackers measure

An increase in this particular biometric is a good thing. visualspace/E+ via Getty Images

Anne R. Crecelius, University of Dayton

Your heart beats around 100,000 times every day. Heart rate is a key marker of cardiovascular activity and an important vital sign. But your pulse is not as steady as a precision clock – nor would you want it to be.

As a cardiovascular physiologist, I measure heart rate in nearly every experiment my students and I perform. Sometimes we use an electrocardiogram, such as you’d see in a medical clinic, which uses sticky electrodes to measure electrical signals between two points of your body. Other times we use a chest strap monitor, like ones you might see on someone at the gym, which also detects heartbeats based on electrical activity.

As wearable technology has grown more popular, it’s not just researchers and cardiologists who are paying attention to heart rate. You might be monitoring your own all day long via a fitness tracker you wear on your wrist. This kind of wearable device uses green light to detect blood flow beneath your skin and deduces your heart rate.

Here are what heart rate and other measurements derived from this biometric can tell you about your body’s health.

Pumping blood where it needs to go

The heart’s primary job is to contract and generate pressure that helps pump blood to the lungs to be oxygenated and then on to the rest of the body to deliver oxygen and other nutrients. Heart rate is simply how fast your heart is beating. Sometimes called a pulse rate, it’s normally presented in beats per minute. You can count your own heart rate by feeling for your pulse inside your wrist or behind your jaw.

When your body demands more oxygen, such as during exercise, heart rate will increase along with the increasing workloads.

While many people are familiar with tracking their heart rate during exertion, the heart rate at rest can also provide valuable information. The two parts of the autonomic nervous system, the sympathetic and parasympathetic, influence resting heart rate. The sympathetic branch helps coordinate your body’s stress response. The more active it is, the higher it dials up your heart rate, preparing you for fight or flight.

The parasympathetic branch of your nervous system is responsible for keeping lots of your body’s functions running smoothly while you’re at ease. Via the vagus nerve that runs from the brain all the way to the abdomen, the parasympathetic nervous system actively slows the heart down to resting values between 60 and 100 beats per minute for the average healthy adult. Without any parasympathetic activity putting the brakes on the sympathetic nervous system’s signals, your heart would beat at approximately 100 beats per minute.

A lower resting heart rate indicates an efficient heart and a higher level of parasympathetic activity. When you’re at rest your nervous system is ideally minimizing sympathetic activity, so you’re conserving energy and avoiding unnecessary stress to the body.

chart of red peaks of a heartbeat at slightly different intervals
The chart of a heart rate reveals tiny differences in spacing between the peaks representing heartbeats. YitzhakNat via Wikimedia Commons, CC BY-SA

Time between each heartbeat

One specific way to understand the balance of the nervous system’s influence on heart rate is to look at heart rate variability, or HRV – the slight fluctuation in the time between each heartbeat. Even if your heart rate is 60 beats a minute, that doesn’t mean your heart is pumping exactly once every second.

Less variability is a sign that your body is under greater stress and that the balance in your autonomic nervous system is tipping toward the sympathetic branch being in charge. Greater variability suggests you’re more relaxed and your parasympathetic nervous system is in control.

For nearly 30 years, scientists have been interested in how to measure and interpret HRV, specifically as it relates to this balance of autonomic control.

The clinical utility of HRV emerged in patients following cardiac events, but researchers are now considering how this measure can help explain patient outcomes in a range of cardiac, endocrine and psychiatric disorders.

More recently, researchers have investigated how to use HRV in athletic training and prognosis of medical conditions.

Several fitness wearables also report heart rate variability, either as a stand-alone metric or used in the calculation of “readiness” or “recovery” scores. Endurance athletes now commonly track HRV as one way to monitor their overall physiological state.

Researchers have started checking which commercially available wearable devices are most reliable and accurate at measuring HRV, which can vary from tracker to tracker. Many of these devices use colored lights, or optical sensors, to measure pulse rate and other variables at the wrist or finger. Unfortunately, the accuracy of this method can vary based on skin type and skin color. It is important that companies include diverse populations in the design, testing and validation of these products to help address potential racial health disparities.

woman doing standing pose on yoga mat
Another health benefit of stress-busting activities can be an increase in heart rate variability. David Espejo/Moment via Getty Images

Nudging HRV in a good direction

One of the biggest influences on heart rate variability is stress; along with increased sympathetic nervous system activity, stress is associated with lower HRV. Stress-reducing interventions, biofeedback and increased fitness can increase heart rate variability. Remember, an increase is good for this metric. Overall, heart rate variability depends on a range of physiological, psychological, environmental, lifestyle and nonmodifiable genetic factors.

The most useful way to consider heart rate variability as a metric is to look at data trends. Are there consistent changes in HRV in either direction? Examine these changes alongside other health factors such as fitness, mood, illness, sleep and dietary intake to see if you can draw any conclusions about lifestyle modifications you may want to make.

In general, the same approaches you would take to lowering resting heart rate can also improve heart rate variability, such as increasing cardiovascular fitness, maintaining a healthy weight, reducing stress and getting sufficient sleep.

It’s important to remember that heart rate variability is the normal, healthy, very slight fluctuation of timing of heartbeats – just milliseconds of difference from beat to beat. More dramatic changes in heart rhythms or the way in which the heart contracts, known as arrhythmias, may signal a more serious condition that requires medical attention.

Anne R. Crecelius, Associate Professor of Health and Sport Science, University of Dayton

This article is republished from The Conversation under a Creative Commons license. Read the original article.