In their virtual world, norns live as cute, clever pets that reason and learn. In the real world, the military has enlisted the technology behind norns to create top fighter pilots. Clive Davidson reports

LIKE A SENTINEL defending the cyber-sky, a graphical representation of a jet fighter flies across a simple, polygonal landscape. Suddenly, off to one side of the computer screen, a second jet appears. The defender responds immediately and a dogfight ensues.

The intruder turns tightly and begins to climb. As it banks, the defender follows close on its tail. Then the intruder levels off momentarily before dropping into a steep dive. As if sensing that its prey will try to pull away, the defender rolls over, aligning its nose with the fleeing aircraft. The intruder is now dead in its sights, but there is no gunfire and moments later both jets vanish.

Peter de Bourcier, who works for the Cambridge software company CyberLife Technology, watches all this and notes down the defender's tactics. Its manoeuvres are typical of an experienced pilot hunting an enemy. Yet this is no human Top Gun, but an intelligent software "creature" that defence researchers hope will one day fly a real plane. Unlike the intruder, which simply performs a set of pre-programmed routines, the creature must work out everything for itself, from how to operate the flight controls in order to stay airborne to the best tactics for outwitting its enemy.

The pilot is one of the most ambitious entities to be built with the techniques of artificial intelligence and artificial life. It has "eyes" that see its world and a neural network for a brain which is washed with virtual chemicals that alter its thinking and actions. Nearly everything about it is controlled by binary genes. It can learn from past experience and reason when faced with novel situations.

Its behaviour is uncannily human, which is why creatures like it are being courted by everyone from high-street banks to defence research organisations as a stand-in for the real thing (see "Virtual banking "). Perhaps most surprising of all, the technology that makes the creature tick is not the product of high-flying academic or defence research. It started life in a virtual pet from an innovative computer game-aptly called Creatures.

Released in 1996, Creatures opens the door to an interactive world called Albia. Here, various virtual life forms eat, mate and play. The player's role is to rear a cartoon-like creature called a norn, and to help it through its encounters. Behind the norn's endearing exterior is a web of fantastically complicated programs.

The man who dreamt up norns is veteran games programmer Stephen Grand. In 1992, CyberLife presented him with a challenge. "My task was to create some software agents that people would enjoy keeping as pets," he says. "It was clear that people would have to have a rapport with their creatures, and thus that they would have to believe they were really alive. The point of no return came when I reasoned that there was no way of fooling people that something was alive. I would have to set out actually to create life."

Grand was the right man for the job. "I've been fascinated by both virtual worlds and biological approaches to AI for twenty years," he says. "In 1992, I suddenly found myself with the opportunity, the skill set and a plausible amount of computer power, so I went for it."

He decided to create his creatures as nature had done, from the bottom up. But rather than wait for evolution to run its course, he fast-forwarded: he gave his animals more than 300 genes, dictating everything from brain structure to biochemistry. Grand did not want to program every minute action of the creatures. Instead, he wanted to make them autonomous in their own world. So he gave them simple instincts and drives to satisfy, such the desires to eat, breed and avoid pain. He endowed them with the ability to learn, and hoped that complex behaviours would emerge.

The norn's brain is a software-based neural network-a vast array of nodes interconnected by a complex of virtual wires or dendrites. Each node has one or more inputs and fires off a signal to other nodes when its inputs reach a certain threshold. Although researchers have been designing neural networks for decades, none had the properties Grand needed. "So I had to invent my own," he says. His solution is a network of 1000 nodes divided into nine "lobes".

As a norn explores Albia, it encounters many objects which it can see, touch, hear, smell or taste. Signals from the norn's senses feed through to its "attention lobe", which contains a node for every kind of object in Albia, from carrots and toys to poisoned plants and other norns. The frequency and intensity of the sensory signals cause the nodes to fire at different rates. An algorithm monitors this firing and directs the norn's attention to the most insistent signal.

Once the norn's attention has been captured by an object, say a carrot, its perception lobe collects the various messages from its sensors (see Diagram). It passes these to the "concept lobe" which, with 640 nodes, is by far the largest. This is essentially a pattern matcher. It looks for familiar signal patterns coming from the perception lobe. The pattern triggered by the size and shape of the object, for example, might tell the concept lobe that it's dealing with a carrot.

Diagram : As the norn moves around it uses up its store of glycogen. As a by-product of this process it produces more "hunger" chemical, which increases its drive to eat. An increase in hunger chemical influences the norn's behaviour. It will, for example, reduce the firing threshhold of the "carrot" node in the attention lobe. This means that if the norn sees a carrot and a toy together, it will focus on the carrot. If the norn had eaten recently, it might have played with the toy instead.

The concept lobe sends a "carrot" signal on to the "decision lobe". This is a small lobe with only 16 nodes, each of which represents a single possible action, such as "go left", "go right", or "deal with the object at hand". If the object is a carrot, the norn will eat it.

One of the valuable properties of neural networks is that they "learn". They are used, for example, to look for flaws in glass bottles. The network is trained by showing it images of flawless bottles, then images of bottles with cracks. The firing thresholds of the nodes are then tweaked until the network gives one output for a perfect bottle and another when it spots a crack. The network's structure and thresholds are then fixed, and it should be able to spot any cracked bottles as they fly by on a conveyor belt.

By contrast, the structure and thresholds of the norn's neural networks are never set in stone. They are continually readjusted by experience. This gives it an ability to learn and respond to new situations based on past experiences.

The dendrites between nodes, for example, weaken and die if they are not used. But every time a dendrite carries a signal it is strengthened by a reinforcing algorithm. When a dendrite dies, the node on the end can send out another tendril to attach to an active node either in its own lobe or a neighbouring lobe. If this new dendrite turns out to be useful then it will be reinforced. This way the norn can forge connections for learning new things without affecting the network's established patterns of use.

This ability to continuously learn and the lobed structure are essentially extensions of what other neural networks already do. The real innovation in the norn is its "biochemistry", which also helps to continually restructure the network. "It seems odd that AI missed out biochemistry when the brain swims in a sea of chemicals," says Toby Simpson, executive producer of Creatures.

The levels of the norn's drives-such as the need to eat and avoid pain-are controlled by "biochemicals", which are really numbers from 0 to 255. Like real hormones and neurotransmitters, these numbers relay messages around the norn's body and brain. They influence the brain's decisions by changing the firing thresholds of nodes in the neural network.

The chemicals are secreted by "emitters" attached to neurons and sensors, and their levels are monitored by "receptors" attached to other neurons and sensors. The higher the level of a chemical, the more pressing is its associated drive. And as the norn travels around Albia it attempts to reduce these drives.

When a norn is born, it has only a few "instincts", such as the urge to wander and to try to eat small things. These instincts are not hard-wired, but learnt by the neural network before birth. So when an infant norn comes across a carrot, it will instinctively eat it. As it does so, emitters release the virtual equivalent of starch. Reactions inside the norn convert this into glycogen (which real animals use to store energy), plus a chemical that reduces the level of the hunger drive.

This does the norn "good" and, accordingly, receptors in its brain that detect the decrease in the hunger chemical reduce the firing thresholds of nodes involved in recognising carrots and deciding to eat them. These pathways will then fire more easily than others and will be strengthened by the reinforcing algorithm.

On the other hand, if the norn eats a poisonous plant, the emitters release pain chemicals. Receptors respond by increasing the firing thresholds of nodes in the concept and decision lobes that link poisonous plants with the decision to eat. These nodes are then less likely to fire, and in time the dendrites linking them die. In this way, the instincts of the young norn are overlaid by appropriate responses to different situations-in other words, it learns by experience.

And this is not the only way a norn learns. The player can also intervene to influence its behaviour by clicking the cursor over "slap" or "tickle" zones on the norn's body, releasing punishment or reward chemicals that inhibit or strengthen certain memory patterns.

As if this degree of complexity were not enough, just about everything in the norn-from the way nodes operate and the structure of the lobes to the locations of emitters and receptors-is determined by genes. Norns have a single chromosome made up of a long string of bytes divided into 320 sections that specify different aspects of the creature. When norns breed, some parts of this code cross over from one parent's chromosome to the other, so the genes mix. This, plus occasional crossover errors and random mutations, means that every norn is different. And, as with humans, a norn's behaviour is a product of the interaction between its unique genetic make-up and experience.

Before Grand began creating Creatures, most experiments in artificial life had focused on replicating individual aspects of organic life. Researchers had used neural networks to model brains, and genetic algorithms to study evolutionary adaptations. But few had tried to create a whole organism with these techniques.

This point was not lost on Dave Cliff, an artificial life expert now at the AI Laboratory of the Massachusetts Institute of Technology in Boston. Warner Interactive called in Cliff to give an opinion of Creatures when it was deciding whether to invest in the game. At first he thought it was a con. He didn't think it possible to have such complex technology running in real-time on a home computer. But when he turned the machine off and made Grand start the program from scratch, the creatures began to evolve all over again in a completely different way.

"It represented a major engineering achievement," says Cliff. "The coupling of the neural network to the biochemistry was genuinely novel."

Grand's hope that employing artificial life techniques would allow norns to display complex behaviours has not only been fulfilled, it has been surpassed. "When you're trying to breathe life into something, you expect it at some point to get up and start having a mind of its own, but it still comes as a shock when it happens," he says. "The first time I felt I was really onto something was when I caught two of my creatures apparently playing ball with each other. I nearly fell off my chair!"

Grand admits that it's difficult to tell if norns behave the way they do for the same reasons that humans do. Nevertheless, their behaviour is human-like enough to have caught the eye of researchers in other industries.

"It was clear that behind the trappings of a strange computer game, there was some complex science going on," says Simon Hancock of the British government's Defence Evaluation and Research Agency. At DERA's flight management and control research department in Bedford, Hancock has been trying to invent adversaries for pilots flying missions in flight simulators. Up to now, DERA has used computer-generated opponents that follow rigid, rule-based systems.

"When we put a test pilot into a simulation of one of our new aircraft, we need to put him in as realistic a scenario as possible," he says. "If the pilot can sense that something unnatural is happening, that distracts from his task of evaluating the aircraft." Hancock hopes that norn technology will provide more realistic enemies that fight with the skill and ingenuity of human pilots.

This is the project that is running on de Bourcier's screen at CyberLife. His machine is evolving generations of virtual pilots which are trying out different flight strategies as they adapt to controlling the digital jet and tracking the enemy. The success of each pilot is judged by simple criteria: at first, by how long it stays up in the air, then by how closely it tracks its prey or evades an attacker and, finally, by how long it holds the enemy in its sights. The best pilots from one generation are then used to sire the next.

Like norns, the synthetic pilots contain a cocktail of biochemicals, but these are not linked to specific drives. Instead, the chemicals, receptors and emitters are just thrown in as wild cards to see what the pilots make of them. It's very difficult to tell if the chemical reactions that have evolved represent fear, stress or aggression, says de Bourcier. Whatever their emotional equivalent, they have helped to create formidable pilots.

After running populations of 40 pilots through up to 400 generations of evolution, CyberLife has software agents that don't crash their planes and can keep targets in their sights for a long time. On paper, these synthetic pilots look as good as human aces, but DERA has not yet put humans through exactly the same test on a simulator. That would be one of the next steps for DERA and CyberLife to take.

The synthetic pilots demonstrate some remarkably human behaviour, such as banking the jet in a turn and rolling over before starting a steep dive. Humans roll over before diving to stop the blood rushing to their heads. The synthetic pilots don't suffer such physical constraints. They have developed this tactic because it helps to keep targets in their sights for longer. The synthetic pilots have also evolved distinctly nonhuman techniques, such as flying the jet in a continuous roll, which can improve stability in some extreme manoeuvres.

Controlling a digital aircraft in the calm of cyberspace is one thing, but could norn technology triumph in a real plane in a real dogfight? The researchers at DERA and CyberLife want to find out. "If these artificial pilots can fly simulated aircraft around in cyberspace, why can't they fly in real life?" asks Hancock.

Most existing unmanned air vehicles are flown remotely by a pilot on the ground. This is not ideal. "First, it is very difficult to fly an aircraft successfully without all the cues from the outside world," says Hancock. "Secondly, if the communications between the ground and the aircraft are jammed, the aircraft is cut off."

One answer, Hancock believes, is to create a synthetic pilot that a commander on the ground could give instructions to and then leave to carry out the mission autonomously. If attacked by an enemy, it would know how to take evasive action. But the implications go further than that. If there is no human inside the aircraft, its design need no longer be constrained by human dimensions and frailties. For example, it could be smaller and be made to turn and accelerate faster. Humans can tolerate a force of up to roughly 9 -a synthetic pilot might find it useful to pull 25 turns.

One thing that the work on synthetic pilots, norns and other projects has demonstrated, says de Bourcier, is that "to develop intelligent behaviour you need a dynamic environment: one that reacts and changes". CyberLife is now employing this lesson back in its virtual world. The company is experimenting with the idea of turning the entire environment of Albia into an autonomous synthetic biological system, with a sun that rises and sets and a gaseous atmosphere that plants and animals breathe. So far, the company has created digital plants that photosynthesise sunlight and digital bees that look for nectar and pollinate plants. Already, unprogrammed behaviours have emerged, such as swarming bees and plants that adapt to different levels of water and shade.

It seems anything is possible. Virtual pets may be fun, virtual ecosystems may be instructive, but if virtual fighter jet pilots can replace real pilots, who knows what human roles these exotic creatures will be taking on next.