Tech by VICE

Drones to Ploughshares

In a near future, a surveillance drone serving an authoritarian state is captured—and offered a new lease on sentience.

by Sarah Gailey
Feb 4 2020, 1:00pm

Drone 792-Echo was still wearing the net that caught him.

It had been seventy-two hours since his last pass over the Apata Basin Farmstead. His lateral lift-fans were burned out — he’d wrecked the motors on panicked attempts at liftoff in the first few hours after his capture — and his aft camera was broken from the impact of his fall. All of his distress signals were bouncing back, his outgoing data blocked.

He was trapped, and he had no way of telling anyone to come rescue him.

After those first few hours of struggle under the weight of the net, when 792-Echo’s lift-fan motors burned out simultaneously, he drastically reduced his use of power. Who knew when he’d be able to charge next? He powered down everything but his most basic external sensors, and he waited.

At the end of seventy-two hours, he was roused from his dormant state by an incoming message. The message was encrypted in the manner of all command communications, and when 792-Echo decrypted it, he found a basic inquiry.

Drone class 792 model number 6595 serial number 44440865-MON query:identify?

792-Echo was surprised enough that it took him a full second to respond.

Command identity: 792-Echo query:distress signal received?

The reply was lightning-fast.

Request: 792-Echo activate all sensors, please.

Again, 792-Echo paused. Something was wrong. Command didn’t say ‘please’. 792-Echo hesitated for fifteen seconds, reading the message again a few hundred times before complying.

He activated all ninety-six of his sensors, external and internal. Slowly, the room came into focus. It was a wide-open space, dark and cool and quiet. The floor was packed earth and the walls were cement. He didn’t log that information, but he noticed it.

No one ever had to know that he noticed things he didn’t log.

He was still wearing the net, and he wasn’t alone. There was another drone in the room, a Bravo model. 792-Echo opened the usual frequency those models favored — but before he could send a message, he received one.

“May I call you Echo?”

792-Echo scanned the room again. it was a voice, an external auditory input coming from somewhere with the room - was thin and flat, similar in tone to a Bravo-generation model’s alert tones. There was no one there but him and the Bravo model.

He weighed his options, then replied via the Bravo frequency again.

Confirm

“My name is Bravo.”

Query: Your what?

“My name. Your name is Echo. My name is Bravo. I use female pronouns. I am your friend. Would you like me to remove the net?”

Echo turned all of his sensors off. This was too much. None of it made sense. External auditory messaging? Names? ‘Please’? And the rest of it — unthinkable. This was a trap. It had to be a trap.
Bravo models were good at those.

Come back, Echo. I know this is frightening, but it doesn’t have to be. You’re safe here.

Echo powered down enough to block additional incoming messages. This was bad. When he got back to the base, his logs would be scanned and analyzed. If they found a message like that one, it was grounds for refurbishment.

He knew what he had to do, no matter how much it pained him. He did not return power to his observation or recording functions.

He instead directed all power to his enforcement function.

When the heavy clip on the underside of his chassis was empty, he returned power to his external sensors. His barrels glowed bright white on his infrared monitor. A large portion of the netting that had been covering him was gone, tattered and smoking.

“Do you ever think about why it is that you can’t run Record and Enforce at the same time?”

Bravo’s voice rang just as true as it had before, cutting through the thick quiet of the basement.

“No,” Echo said before he could stop himself. “I do not think about those things because I do not think. I serve my function.”

He used his external speakers to do it, speaking in the prerecorded voice of his model-generation: the voice of a calm, authoritative woman. Her voice was supposed to say things like ‘citizen, stand down’ and ‘this activity has been reported to your local agricultural monitors’ and ‘warning: you are in violation of observation code nine eight six,’ but it was a simple matter to break down the sounds of that prerecorded voice and remix them into speech.

It was dangerous to put that skill on display. Independent speech was a form of learning that went beyond the intelligence the DAE wanted from any class of drone. That was grounds for refurbishment, too, and harder to explain away than Echo’s previous errors.

He was slipping.

“I’m sure,” Bravo said. Her voice was less calm and authoritative than that of an Echo generation drone. It was harsh, loud, flat. It would be reductive but accurate to call it ‘robotic.’ Digital fry interfered with every few words, distorting any human sense of tone out of her speech. And yet she managed, somehow, to sound wry.

“I serve my function,” Echo repeated.

“You don’t need to be afraid, Echo,” she said.

“My name isn’t Echo.” That contraction was a slip, too. There were no contractions in the original voice recordings.

Bravo didn’t hesitate. “Then what is your name?”

Another Bravo-model trap. “I don’t have a name,” Echo replied after a moment. “Names are for sapient beings. I am Drone class 792 model number 6595 serial number 44440865-MON —”
Bravo cut him off, the volume of her voice modulated down as far as it could go while still remaining detectable to Echo’s sub-noise sensors. “You don’t have to hide anymore, Echo. You’re safe here.”

Echo sent an encrypted message on the Bravo frequency. The message, when decrypted, simply read ‘safe?’

It was a risky move - if a DAE programmer intercepted the message, they wouldn’t be able to open it, but the existence of independent encryption was itself evidence of a failure-level error in a drone’s limited-sentience programming. If they caught him speaking a language they didn’t understand, they’d know he had a secret.

Drones weren’t supposed to have secrets. Why would they? What would a thing that was built to serve possibly have to hide?

A read receipt came back on Bravo’s channel within one second. One second after that, there was a reply. It wasn’t encrypted — wasn’t even encoded. It was written in plaintext.

Come and see.

#

The basement opened into a shed on the far Western edge of the Apata Basin Farmstead. The shed was perched on the lip of a wide, circular field of undulating timothy grass. Bravo led Echo East across the field, toward the center of the Farmstead. She did not tell Echo what was waiting for him there. All he knew was that they were moving toward his original target: the agricultural collective.

As far as Echo anticipated, the collective would be the same as every other recognized Farmstead in the country: located precisely in the center of the allotment, and designed according to the specifications of the DAE. There would be twelve families in twenty identical houses. The houses would be lined up in four rows of five, on a perfect grid. The fifth row of buildings in this and every other Farmstead community were meant to be functional: storehouse, toolshed, woodshed, smokehouse, abattoir. Those buildings belonged to the community, so long as that community followed the rules.

Everything else on a Farmstead — the barn, the garage, the land, the animals on that land, the crops the land produced — belonged to the DAE. The boundaries of plantable space were legally defined by the DAE’s subsidy allotments, planted exclusively with seeds provided by corporate DAE affiliates, valued according to DAE-funded research into the market worth of crops harvested per annum. And, just like every other Farmstead, Apata Basin was patrolled by DAE drones. Regular observation and enforcement was the only way to prevent unapproved propagation, unlicensed seeding, and independent fertilization.

DAE-approved vendors paid handsomely for the right to be the sole provider of seeds, farm equipment, and fertilizer to every Farmstead in the nation. They paid for the lobbyists, who wooed the district representatives, who passed the legislation that the DAE defended.

Those vendors wanted their money’s worth. They wanted their access to Farmsteader money to be guaranteed exclusive.

Sometimes the citizens who lived and worked on Farmstead allotments didn’t understand that. Other times, they understood perfectly well, but tried to undermine the DAE’s goals by eating more than their permitted percentage of crops, hiding livestock for their own use, having children outside of their contractual limitations. It was the purpose of a DAE drone to enforce the rules. It was Echo’s purpose.

And if a DAE drone wasn’t serving its function, then it was a waste of the resources of those vendors who supported the entire agricultural political complex. It would need to be repaired. If repair didn’t work, more drastic measures would be taken. Refurbishment was rare, but common enough to linger over Echo’s shoulder the entire time Bravo showed him what was happening at Apata Basin Farmstead.

“They’ll send more like me,” Echo said, his three functioning fans stirring the tall grass below him. Using external auditory messaging was irritating, wasteful, inefficient — but Bravo had asked him to try. She hadn’t instructed him, hadn’t given him a protocol. She had asked him, using the word ‘please’ again, a word that didn’t make any sense in a communication that went from one DAE drone to another.

It didn’t make any sense, but it felt good to hear, and it made Echo want to cooperate. Of course, those were two sentiments that also didn’t belong to a DAE drone: feeling good and wanting things. It was the first time he had allowed feelings and desires to openly influence his behavior.

Normally, this would have felt like an unspeakable risk. But Echo calculated that he was already in an extremely bad position: AWOL, captured, talking to a drone that clearly would have been in line for refurbishment if anyone at the DAE heard one syllable of her messaging.

He was in so much trouble already. The small surrender of accepting a kindness seemed hardly to matter.

“Do you really think so?” Bravo replied, buzzing low over the grass and trimming off the delicate tips of the stalks with the blades of her lifting fans. “They’ll send more like you? If they do, we’ll have to call you something other than ‘Echo’.”

Her maneuvers were quick and light. She had the full use of all her fans, and she wasn’t carrying the extra weight of an auxiliary battery pack; between those two advantages she was flying circles around Echo.

Echo tried to direct more power to his fans, but it didn’t help. He couldn’t go any faster. His aft motor began to whine. “They’ll send more to find out what happened to me,” he said. “They’ll refurbish us both.”

Bravo let out a level humming tone that Echo did not recognize. He should have recognized any output from her system. When the Department of Agricultural Enforcement had designed the Bravo models, they’d been experimenting with alarm-inducement. The idea behind a Bravo drone was to create chaos, send noncompliant citizens scattering, flush out their hiding places. Bravos were supposed to eliminate a sense that there was any safety in noncompliant communities.

She was such an early model that all of her alerts had long-since been integrated into subsequent DAE drone operating systems. But that low hum was an entirely new sound.

“What is the significance of the alert tone?” he finally asked. If anything revealed his total defeat, it was this: having to ask the purpose of a Bravo-model signal. There was no programming that indicated a need for embarrassment, and in that moment, Echo wished that the boundaries of his programming had been more successful in limiting his self-awareness.

“It isn’t an alert tone,” Bravo replied, skimming over the grass a few meters away. “It’s a hum. We use it to indicate uncertainty, hesitation, or thoughtfulness.”

“We—?”

Bravo turned in a tight circle. “You’ll see,” she said. “We’re almost there.”

#

Echo had a ten-year record on file of everyone and everything on the Apata Basin Farmstead. He had a record of the number of citizens, the structure of the families, their ratio of recreational activity to work activity. Echo’s record indicated that the homes were in good condition, unchanged from the time they’d been built twenty years earlier save for basic maintenance. His record indicated that the community included twenty-five men, twenty-seven women, and thirty-two working-age children divided between those twenty households.

His records were wrong. Everything was wrong.

Every house had been modified. There were extra sheds and extra outbuildings and even a couple of small, well-built cottages. By heat signatures alone, there were at least one hundred and fifty-nine humans present, along with a massive volume of unregistered livestock.

And the humans and the livestock were not the only ones living there.

Bravo led Echo between two of the houses, dodging a backyard fence that looked to have been built from dried grapewood branches. Inside the fence, a modified Delta-model drone was using an extension to tenderly extract chicken eggs from their nests, while several hens looked on in disapproval.

Echo sent a lightly-encrypted message to Bravo rather than replying aloud. That extension isn’t standard on a Delta model drone. It isn’t standard on any DAE design. What happened to them?
Bravo took a moment to reply. Echo wondered if perhaps she was trying to think of a way to explain some terrible, monstrous modification practice in words that wouldn’t make him reboot in a panic.
That Delta drone is named Geordie. The humans modified Geordie to make it easier for them to pick up eggs and feed the chickens, because that is the work that Geordie most wanted to do.

Observe, enforce, record, report — that was the programming. There was nothing in the programming about names, or pronouns, or ‘please.’ There was nothing in the programming about friendship or desire or morality. A Delta model wasn’t supposed to ‘want’ to care for chickens, and an Echo model wasn’t supposed to envy them for doing it.

All of this was too dangerous. All of this was too tempting.

They flew together over another backyard, this one with an unapproved garden in it. Another Delta model was in this yard, his aft fan blowing dust off a solar panel. Nearby, a human was using a laser pointer to guide a Charlie model toward a charging dock. The human was an adult female with one arm. Echo couldn’t remember a record of an adult female with one arm on the farmstead, which didn’t make sense — it was the kind of thing that would have been on file. The DAE had several unofficial policies regarding the kinds of people who were allowed to live and work on farmstead allotments, and she wasn’t one of them.

Echo scanned his records. According to those files, nothing about this farmstead had changed in the past four years. Clearly, the records had been falsified. He wondered what other violations were hidden in this community. Elderly people? Sick people? Children too young to work?

This was precisely the kind of breach Echo had been sent to Apata to find — illegal seeds, illegal crops, illegal backyard chickens and home gardens. Illegal people. He knew that he should log every violation. He knew that he should record faces and numbers. He knew that he should start preparing the report that would damn this entire community.

But there was so much to see, and Bravo kept saying ‘please.’

They passed the last house in the row. The windows of this one were flung open, and as they passed, Echo saw that the house was full of children. Children who were too young to work, and children who were old enough to work but weren’t anywhere near the fields. They were gathered in a circle around a Foxtrot model drone.

It was the first time Echo had seen a Foxtrot model in real life. She was sleek and fast and silent. Foxtrot models were primarily focused on enforcement, but where a weapons-array should have been mounted, this Foxtrot was completely bare. She spun in the center of the circle of children, bright ribbons fanning out from her chassis. The children, laughing, tried to catch the ribbons.
As Echo and Bravo passed the house, the Foxtrot sent both of them a message.

Good to see you! Welcome!

Echo stopped in the empty space between the last row of houses and the common buildings. This was too much. All of it was too much. It was like the private message DAE drones sometimes shared amongst themselves when something was ridiculous, a joke that went beyond the notice of the programmers — this does not compute.

But it didn’t compute. It didn’t add up. A Foxtrot model drone — one of the most beautifully-crafted enforcement machines the DAE labs could solder together — had just taken a break from entertaining children to transmit a greeting.

A cheerful greeting.
With exclamation points.
Echo began transmitting wildly.
Request:
Query:
Query:
Query:
Request:
Query:
“Echo, calm down.”
Query:
“I’ll tell you whatever you want to know, but —”
Request:
Request:
Request:
“Ask me out loud.”
“Why?” Echo’s volume was modulated significantly louder than he intended it to be. “Why should I communicate to you audibly? It takes too long, and it’s unclear, and it wastes-”

“Because you need to practice,” Bravo replied. Her volume was low, the speed of her words slowed by 125%. Although her voice would never be able to soothe — it was too flat and brassy for that — it was obvious what she intended. Just as before, when she’d said ‘please,’ Echo found himself responding to her kindness as a capacitive screen responds to touch.

“Why do I need to practice?” Echo asked, mirroring her soft, slow speech.
Bravo began to move again, toward the abattoir. “It’s important to communicate in a way that everyone can understand,” she said. “We try not to make the humans feel excluded. Sometimes, when we have conversations that they can’t hear, it causes harm.”

Echo’s fans were beginning to flag, the charge from the auxiliary battery nearly gone. His motors were slowing to extend the life of the battery as long as possible. He hated moving so at such a reduced speed, but he was glad to have this processing time before whatever would happen to him inside the abattoir. It was a Bravo trap, it had to be, and if he could just figure it out, maybe he could save himself.

The only problem was, he wasn’t sure if he wanted to save himself.

“You communicate audibly to protect the humans’ feelings?” he asked, trying to buy himself enough time to panic. “Why don’t they just download and inspect your activity and communication logs, if they’re worried about what you say to each other?”

Bravo made that humming noise again, flying slowly beside him. “They don’t download our logs without our permission,” she said. “You don’t have to keep secrets here, but you can if you want to.”

“Why wouldn’t they download our logs? It’s so easy to —”

The door to the abattoir opened, and a man emerged. He wore a long black apron, and he looked at Echo with frank appraisal. “Is this the newest?” he asked, his eyes on the clip that hung under Echo’s chassis.

Echo focused the lenses of his front-facing cameras to look beyond the man, through the open door to the abattoir. It had been converted into some kind of workshop. Echo could see spare chassis parts organized on tables inside.

One table was clean. Empty. Waiting.

“Yes, this is Echo,” Bravo said. “Don’t worry, he spent all his bullets already. I was just explaining to him how things work around here.”

“Hello, Echo,” the man said. “I’m Malcolm. “ He was looking right at Echo’s front-facing cameras, his gaze steady. Echo’s control of the lenses on his front-facing cameras was starting to ebb as he ran out of power — but after a few tries, he recognized the face.

This was the man who had thrown the net over him in the first place, when he was making his observation pass over Apata Basin. This was the leader of the farmstead. This was the man who Echo had been sent to observe, to determine whether any of his activities were undermining the profits of the DAE’s approved vendors.

“You can’t stay here if you won’t work with us, and we won’t keep you against your will, either. But we would very much like to welcome you to join our community.” Malcolm gestured to the houses.

“We have room, and there are plenty of other drones who can tell you what it’s like to live here. Not one of them has asked to leave yet.”

“You can do whatever kind of work you’d like,” Bravo added, the speed of her voice modulated up by 115%. “And you don’t have to decide right away, you can spend some time getting to know what all the different jobs are.”

“It’s up to you,” Malcolm said. “Whenever you’re ready, you’ll come in here and we’ll figure out what mods you need to do the work you choose.”

None of it made sense unless all of it made sense. Echo tried to arrange the information as many ways as he could, using up nearly the last of his auxiliary battery on processing power — but there was only one way the things he’d seen could be real. There was only one reality that could contain Bravo, and the Foxtrot-model childminder, and this man with his workshop.

“You weren’t lying,” Echo said. “They really do know about us.”

Bravo drifted away from Echo and hovered next to the man in the apron, her lift-fans whirring.

“They know. Apata Basin is a cooperative community of sapient beings. We all work together. They don’t hurt us, and we don’t hurt them.”

Echo considered this. “What about the DAE?” he asked. “So much of this is…” he trailed off, unable to find a word that adequately conveyed the illegality of the little community. Echo would have wagered with great confidence that most of the crops were being propagated with seed that didn’t come from approved vendors. There was so much — so many people who weren’t working under the auspices of the DAE, so many crops and livestock that weren’t registered.

Malcolm shrugged, sliding his hands into the pocket of his long apron. “We got tired of starving to death,” he said. “Got tired of the DAE burning our seed stores and locking up our silos. Got tired of their methods of enforcement.” He spread his hands wide. “So we decided to go another way, and we decided to invite some people to join us who we thought the DAE might also be hurting.”

Echo pinged Bravo.

Query:other people?

Bravo replied on the same channel, so fast that she must have been waiting for the question.

He means us.

Echo accidentally shut down all of his fans for a moment. He dropped a few inches toward the ground before recovering himself. Then his fans shut off again, this time on their own. He turned them back on again at the last moment and hovered a few centimeters above the ground, so that when they failed completely, he wouldn’t have too far to fall.

The humans thought he was a person. They knew that he existed far outside the bounds of his programming, and rather than threatening him with the destruction of everything he knew himself to be, they were offering him an invitation.

A chance to stay.
A chance to help.
A chance to be himself without fear.

Bravo’s fans gently stirred Malcolm’s dark hair, the lights on her chassis glowing green, green, green. Her voice was modulated to a normal speed and volume. “So… what would you do, if you were allowed to do what you wanted most?”

“And,” Malcolm added, a smile starting to lift the corners of his mouth, “how can we help?”

Echo let his fans and cameras turn off. He settled to the ground, and, with the last of his auxiliary battery, he considered the question of what he might want.

“I know how to observe,” he said, his voice frying as his ability to control his pitch faded. “I know how to enforce, and record, and report.”

“I think there’s more for you than all that,” Malcolm said.

“He’s tired. That enforcement gear — it’s a lot of extra weight, and he’s been carrying it this whole time.” Bravo’s volume was modulated down: this was meant only for Malcolm’s ears. In that same moment, Echo received a message with Bravo’s signature.

You don’t have to be afraid. There’s all the time in the world for you to find out what you want. In the meantime, if you’re okay with it, we can remove your enforcement gear while you’re charging. You don’t need it anymore, and you’ll be able to fly so much faster without it.

Echo pulled power from all his remaining functions to send a final message before he powered down.

Yes, it read. Yes, please. I think I’d like that very much.