Creativity in Humans and Machines

“Imagination is more important than knowledge. For knowledge is limited to all we know and understand, while imagination embraces the entire world, and all there ever will be to know and understand.” A. Einstein

As humans, we are prodigies of imagination. But what about machines, driven by AI algorithms? Could they also possess creativity that rivals or even surpasses our ability for originality? Can they imagine theories, designs, and art as proficiently as we can, or perhaps even better? In this article, we will plunge into this captivating topic, exploring the intriguing possibility of AI-driven creativity.

Children are the ultimate creative minds.

Children are naturally endowed with a rich and vivid imagination. Their minds are filled with boundless fantasy, allowing them to perceive the world in ways that often surpass the limits of reality. They draw impressive pictures, abstracting reality and transforming it into their fantasy world. Children can transform mundane objects and situations into extraordinary and magical experiences through their fantasies. They can effortlessly turn a cardboard box into a spaceship, a stick into a wand, or a pile of sand into a majestic castle. Their ability to create fictional worlds and characters is unmatched, as they blend reality with their imaginative concepts, bringing to life stories and adventures that unfold in their minds.

They can create fictional characters full of unique personalities. By swapping roles in a play, children acquire a deeper understanding of themselves. Their imaginative capabilities allow them to transcend reality, breaking the bonds and limitations of mere observation.

It is important to note that children’s knowledge is inherently limited. In their early years, they have a remarkable capacity to absorb information like sponges. However, their utilization of this knowledge differs from that of adults. Children possess a synthetic mind, prioritizing imagination over analytical thinking. Rather than focusing on uncovering existing qualities, young children excel at envisioning new possibilities and qualities that surpass what already exists.

Artificial Intelligence (AI).

Fig. 1 Artificial Neural Networks are composed of computational elements arranged in layers, mutually connected where input from one layer feeds into another layer.

Can we artificially replicate the creative mind of a child? Let’s first look at what we already can achieve with Artificial Intelligence (AI) and point out the limitations of this technology, primarily related to the capacity of AI to explore outside the territory of the already known.

AI is most commonly implemented through Neural Networks Algorithms (NNA). In NNA, multiple connected units inspired by biological neurons serve as the data-manipulating components that transform the information (see Fig. 1).

The NNAs employed for image and text manipulation, which are popular and successful, are deterministic algorithms. This means you will consistently obtain the same result regardless of how often you repeat the input-output operation. No dice are tossed.

Deterministic NNAs excel at interpolating data. The jargon in the field says that a successful NNA can “generalize.” NNAs are currently the most versatile and powerful interpolation machines available. NNA requires a large data set of correct solutions to train the network. This set is called the training set, and though it can expand by adding further examples to feed the network, it is fixed once the training is accomplished. This usually happens when a particular objective function reaches a pre-determined minimum (or maximum) value. It is a static machine that can’t change its point of view, introduce its own bias, or add configurations not present from the start.

The Cybernetic bet.

Many years ago, as an undergraduate student, my professor of Cybernetics (yes, that old-fashioned word ) introduced me to the Cybernetic Bet*.

The goal of cybernetics is to control a system that operates in a natural environment that, by its dynamic nature, is changing in an unknown fashion. The Cybernetic Bet improves control by introducing a certain amount of randomness.

As an example, I take a plant watering system.

In this scenario, many valves regulate each plant’s water amount. In general, you know where the plant is and the type of plant and decide to fit together all parameters at the start, such as the water at the sprinkles, the pressure at the source, and the type of plant. However, some plant variations, the ground, and seasonal differences can’t be foreseen entirely. To solve this uncertainty in the system, the cybernetic specialist introduces uncertainty in the regulating mechanism. This can be done by randomly changing different amounts of water passing through the sprinkles. For example, allowing the parameters to be proper for only 90% of the time. In this way, we can explore the values of the “sprinklers” space that might, who knows, allow better growth. However, allowing uncertainty in the parameters, we expose our plants to the risk of 10% overwatering or underwatering. Suppose the Cybernetic Bet machine is equipped with some ratchet device that allows the inclusion but not the release of the better sprinkler parameters once they are discovered. In that case, you are in the business of improving, step by step, your irrigating system in time, adapting to the plant needs and variations of the environment.

The analogy with the irrigation machine highlights the necessity of random processes in adaptive systems to introduce novelty and foster improvement. By incorporating randomness and exploration into the system’s decision-making process, it becomes more flexible and capable of adapting to changing circumstances.

• In Machine Learning, the Cybernetic Bet is called the Epsilon-Greedy Algorithm.

The solution space.

The solution space serves as a valuable tool for understanding the input and output of a Neural Network Algorithm (NNA) and provides insights into the concept of generalization versus creation. By representing input and output variables graphically, typically in a 2D space, we can explore the behavior of the NNA.

Fig. 2A) The dots in this 2D representation of the solution space exemplify a data set usually part of a multidimensional space. Each dot is one case of a training set, for example, a word, a picture, or a token we want the algorithm to classify. These are discrete elements occupying only a single x,y location. The NNA can assign, given a partial input, an output that will appear near to the most similar of the elements in the training set, recognizing in this way the associated figure. The blue dot, far outside the training set area, is not easy to reach by conventional NNA and requires a significant leap of imagination. The example shows a child-drawn cartoon of a cat as an example of a fantasy cat representation.

Fig. 2B) Here, the dots are clouds to indicate the probability of a particular figure instead of a specific single instance. This forms a continuity of possibilities, as, for example, is implemented in a Variational Autoencoder.

Consider Fig. 2A, which displays a set of dots arranged in a 2D space. With an appropriate representation, these dots can describe words or figures. Imagine that these dots represent the learning set fed into the NNA for the learning process. These points, which we already know, form the training set for the NNA.

Let’s consider the scenario where the NNA is presented with limited information as input. In this case, we can ask the NNA to fill in the gaps and suggest the most plausible complete data set. For instance, if we provide the NNA with a partial picture of a cat, we expect it to transform that partial picture into a complete representation of a cat. This process can be seen as an interpolation of a cat within the solution space, given the previous knowledge.

It’s important to note that the complete picture of the cat generated by the NNA will not match any specific cat from the training data set. However, it will bear similarities to many of the cats within that set. In the picture, the newly discovered cat produced by the NNA will lie somewhere within the area occupied by the training set, indicating its resemblance to the examples it has learned from.

In this “toy model” of NNA implementation of AI, creativity is positioned outside the area represented in Fig. 2A, where the examples in the training data set are located. Most AI software today does not access this outside area. The Cybernetic Bet machine might suggest a way to escape the boundary of a rigid deterministic AI. It will need a level of sophistication much higher than our plant watering device. However, such an improvement to AI will come at a cost. Like our plant watering machine, the new AI-fitted aleatoric engine will eventually make mistakes.

In the book “ The Hitchhiker Guide to the Galaxy “ Douglas Adams describes a fictional propulsion device called the Infinite Drive:

“Once the drive reaches infinite Improbability, it simultaneously passes through every conceivable point in the universe. An incredible range of highly improbable things can happen during the ride.

The drive, at the same time, diverges in the impossible and converges in the improbable.”

Every possible and conceivable location can be reached by an Infinite Drive starship, with some drawbacks: highly improbable events can take place during the journey, like “the transformation of a pair of guided nuclear missiles into a sperm whale and a bowl of petunias”.

The AI of the future, like the Infinite Drive, once enabled by a creative mind, will wander in the space of possibility. Just like the human mind, it will eventually come to solutions that can be both imaginative and absurd.

Meaningful Creativity.

From the example above, as reported in Fig.2A, it might appear that to achieve creativity is just enough to randomly pick a point outside the “comfort zone” of the training set. It is not so easy. Creativity, like consciousness or intelligence, is an elusive concept. To achieve an outcome we can call creative, it must be meaningful. There is no picture isolated in a human mind: any image, text, or sound needs to relate in a particularly human way. The invention casts a network of such relations in the human mind unexpectedly, as such relations are not available in any training set but are still meaningful for us. Just like humans, AI, to achieve real creativity, must be capable of selecting between these meaningful (to human) relationships.

AI vs Children.

What about humans, do they are equipped with some sort of “infinite drive”? If we look at our kids, they definitely perform better than any AI in terms of creativity. A child can create a remarkable cat representation by envisioning a fantastic face with rounded features, semi-closed eyes, long whiskers, and pointed ears (fig.2A). This astonishing performance can be achieved with minimal exposure to pictures of felines.

Children can swiftly generate imaginative and novel ideas, transcending the limitations of existing knowledge and training.

Subjectivity of knowledge.

According to the philosopher Schopenhauer, when we observe reality, we inevitably contribute a personal component to it, resulting in a unique representation for each individual. This act of subjective interpretation is a form of creativity. Humans do not perceive the world objectively but rather through the filter of their interpretation, which includes a synthetic personal element. We cannot help but inject fantasy elements into our perception of reality. Unlike machines designed for mere data collection and analysis, we are no collect-and-fit machine, but we subjectively represent the world.

This subjectivity of human perception bestows upon us an innate capacity for invention. Our ability to interpret and shape reality creatively is a distinctive feature of our minds.

The creativity we infuse into our knowledge adds value to our understanding of the world. It allows for the exploration of new ideas, the formation of unique perspectives, and the creation of novel concepts.

While scientific endeavors strive to eliminate subjectivity through repeated experiments, adoption of the universal language of mathematics, and the confutable realm of scientific theories, inherent subjectivity remains a valuable aspect of our imaginative faculties.

Interpolative vs creative process.

The human mind operates in an imaginative rather than interpolative manner. When we observe reality, we interpret it through the lens of our imagination, actively immersing ourselves in the instances we perceive. This imaginative process introduces room for mistakes and errors. Similar to the Cybernetic Bet concept, humans make mistakes after a certain number of trials. However, the human imagination can transcend the limitations of a “training set” that AI struggles to overcome.

During the “AI winter” of the early 90s, attempts were to enhance Neural Network Algorithms (NNAs) by incorporating randomness. I worked on a type of NNA that utilized Genetic Algorithms (GA), inspired by nature. In GA-NNA, the connections between artificial neurons are represented as a genome — a list of data that evolves and moves through the solution space — according to specific rules, generation after generation. GA introduces a creative element to the learning process as it explores beyond predefined boundaries. However, there is a drawback to using GA for learning: it is slow, even with the faster computational speeds available today.

So there is no randomness in AI algorithms? Absolutely not. Some of the most potent and innovative AI technologies adopt random numbers in their arsenal of tools. One example is the “Variational Autoencoder”. Such an algorithm creates a representation of, for example, a picture instead of a single point in the space of representation (see Fig. 2B) as a cloud of probability. This allows an uninterrupted exploration of the space of the possible images and the synthesis of some not present in the training set. But still, we are far from seeing a castle in a pile of sand or a missile become a whale in space.

It is often said that knowledge of something is the ability to replicate it. In other words, when we truly understand something, we can reproduce or recreate it. Might it be that we do not understand creativity in the first place, which is why we can’t replicate it in a machine?

Human beings have the capacity to grasp and replicate knowledge, which extends beyond mere interpolation. While AI systems excel at interpolation, humans possess the imaginative capability to go beyond interpolation and engage in creative processes.

What is the point of creativity, after all?

Creativity uniquely influences adaptability and survival within a changing and unknown environment. Machines like autopilots for airplanes or cars do not necessarily require creativity — who wants a creative autopilot after all — they rely on predetermined algorithms and rules to perform their tasks efficiently and safely.

However, creativity becomes crucial in complex and uncertain environments where the rules and conditions are not well-defined. This is where adaptive systems, whether human or machine, must explore new possibilities, generate innovative solutions, and adapt to novel circumstances. Creativity allows for developing ideas, perspectives, and approaches that can address unforeseen challenges and exploit untapped opportunities.

The subjective component of human knowledge, as highlighted by Schopenhauer, does limit our comprehension of reality. Each individual perceives and interprets the world through their personal view, influenced by their experiences, beliefs, and biases.

Starting from the same dataset and contrary to AI, humans can extract very different types of information. Subjectivity is a double-edged sword that we have to cope with. We can embrace our subjective view or minimize it to transform knowledge from subjective to objective. The scientific method and artistic imagination reflect such a duality.

The scientific method offers a systematic approach to minimize subjectivity and enhance the universality of knowledge by relying on replicability, empirical evidence, and the potential for refutation. The scientific method aims to uncover objective aspects of the natural world that we can all understand uniformly and predictably. It seeks to establish theories and models that can withstand rigorous testing and scrutiny, providing a more reliable and objective understanding of reality.

On the other hand, art embraces subjectivity and explores the depths of personal expression and interpretation. It allows individuals to tap into their emotions, intuition, and imagination to create something unique and meaningful. Artistic endeavors often prioritize self-discovery and the recognition of individual perspectives, offering insights and experiences that may transcend conventional scientific understanding.

Like children who see a sand pile as a castle, human adults can transform the same data set in science or art in their imagination. That is a wonder we are far from reproducing in any AI.

Regarding the acquisition of knowledge, both humans and machines have limitations. The available “training set” for both is finite and constrained by the data and experiences they have encountered. However, humans possess a unique capacity for introspection, doubt, and imagination. This gives us a potential for exploration beyond the boundary of any training set. By creativity, we can imagine any possible scenario and reach part of the solution space impossible for AI to fathom.

The power of doubt allows humans to question existing knowledge, challenge assumptions, and envision alternative scenarios. It enables us to explore the realms of the possible and the impossible, speculate, and imagine potential futures. This cognitive ability, coupled with our innate creativity, helps us adapt and navigate in complex and unpredictable environments.

While machines can learn and process vast amounts of data, they currently lack humans’ inherent imagination and creative thinking abilities. The future of AI is to endow machines with creativity, allowing them to be sometimes wrong. Critical behavior makes us self-confident in our capacity to fail, a necessary skill for future AI. The capacity to switch from the rigorous application of well-tested, objective rules to a subjective way of expression is still far from AI. It is the biggest challenge in this field for the years to come.

AI as a tool instead of a means to an end: augmented creativity.

Although AI is not creative, it can still be a tool for many applications involving creativity. AI tools emerging in the market are the foundation of a new type of creativity, “Augmented Creativity”. Just like a craftsman uses technology to shape his work, AI finds its place in the creative process. In this article, for example, I tested the use of ChatGPT. It is interesting to see how I proceeded, so I will outline my way of using such an application.

I first wrote a draft of the article, and then I submitted it in full to ChatGPT. I was not happy with the result. It was a confused rearrangement of my text, and ChatGPT could not grasp the point of the discussion. So I tried to feed in only paragraphs at a time. The result was much better. As I am not a native English writer, the suggestions from ChatGPT were helpful. Though I still needed some editing to get what I wanted, the application is definitively a tool for writing.

From this practical example, I conclude that AI has space in creativity but not in creating. The creative process, where the idea comes to mind, is still human and can’t come from AI. However, through the human filter, AI suggestions can be helpful.

In other creative fields, like visual art, we see AI software, like DAL-E and Midjourney, that, starting from a text message at the prompt, visualize images inspired by such message. Engineering such a prompt to visualize the picture we have in mind might be a difficult task. Images in the training set are associated with text that describes the image. By introducing text in the input, the user tries to reverse the process to get a picture according to its description. There is both a component of randomness and lack of unicity in the process that create many possible outcomes, hence the possibility for the user to pick the right one for her project. Please note that the randomness here is not the kind I described above when identifying the limit of the solution space. Here, randomness is limited to the choice of starting condition for the algorithm between the many training set images. Once the text at the prompt is chosen (by the user) and the starting image randomly picked (by the program), the algorithm proceeds deterministically to the solution image.

While AI cannot generate the initial spark of creativity or come up with original ideas on its own, it can undoubtedly augment and enhance the work of human creators. By incorporating AI suggestions into your writing, you can refine your ideas, improve the flow of your text, and even explore alternative perspectives.

AI is likely to become a valuable part of a writer’s toolkit, alongside other tools that have revolutionized the creative process, such as access to information on the internet and various publishing platforms. As AI technology advances, we can expect it to play an increasingly significant role in assisting creatives across multiple fields.

In conclusion, while AI cannot replace the human aspect of creativity, through the collaboration between humans and AI, we can leverage the strengths of both to push the boundaries of what we can achieve. The essence of the creative process, the capacity of the human mind to explore, is still far from being successfully modeled by a computer program.