Imagine that you are sitting on a bus. A few seats ahead of you, you see a person you think you recognise. Perhaps it is an old classmate? Using your mobile phone camera, you take a photograph of her.

Don't look now… Pic: Supplied

Using off-the-shelf face-recognition software combined with substantial processing power accessed cheaply through cloud computing, you search for publicly available images (such as on Facebook) matching the image you captured of the woman on the bus.

You find out that the person is indeed your old classmate, and you can also access information such as her address, phone number, birthday and perhaps income.

This scenario is not science fiction. It is reality.

Face-to-data (F2D) refers to at least partially automated processes for accessing personal information about a person based on an image of that person’s face.

While face recognition, and the legal issues associated with it, have gained attention for some time now, recent ground-breaking research by a team of researchers from Carnegie Mellon University highlight there is a new set of privacy concerns

The advancements in face recognition technology, combined with the development of cloud computing and widespread posting of images linked to names on social media sites mean accessing a person’s private information has become a whole lot easier.

The Carnegie Mellon University team have run a series of experiments. Using a search tool, they have built a database of images and names collected from publicly available Facebook profiles.

They then captured images of consenting students and ran those images through the off-the-shelf face-recognition software, linking in the data gained from the Facebook profiles. In the test, about a third of the students were identified.

By mining public sources, including Facebook profiles and government databases, the researchers could identify at least one personal interest of each student and, in a few cases, the first five digits of a Social Security Number.

The scenario of confirming the identity of someone on a bus is of course only one possible way F2D technology could be used.

Other obvious uses include governmental use for surveillance purposes, and business use such as, data gathering (for example, financial position and address) relating to customers entering a shop.

If we change the scenario on the bus so that the person being photographed is only 11 years old and the photographer is a paedophile using F2D technology to identify, track and stalk the girl, the scenario is suddenly not appealing at all (if it ever was).

Like (almost) all technologies, F2D can be used both for good and for evil. This should not be ignored. We must at least consider regulating its use.

We must ask to what extent the Government’s current privacy reform will protect us against F2D – could it be that the “new” privacy law is outdated already before it has taken effect? 

Do we need to go further? Perhaps with F2D we have finally reached a stage where the technology in question is so ‘creepy’ that it does not matter whether it is merely being used for ‘harmless’ curiosity, maybe it should quite simply be banned?

In the end, F2D may be best viewed as just another example of how technology is developed based on what technology can do. Maybe we now need to shift the focus to what the technologies we develop should do?

Comments on this post will close at 8pm AEST.

Most commented


Show oldest | newest first

    • Gavin M says:

      05:39am | 01/11/12

      Ah, “collected from publicly available Facebook profiles”. And that’s the point. If someone voluntarily lives their life fully exposed to the world, then that’s their choice. It’s simple really, if you value your privacy, then live your life privately. This is a non issue for most of us, except for the fact we get a laugh from the fact that people can set themselves up and then protest that it’s not fair.

    • Tubesteak says:

      06:49am | 01/11/12

      +1 well said

    • daniel says:

      08:48am | 01/11/12

      Top post Gavin.

    • Phil says:

      09:00am | 01/11/12

      Not just publicly available data.
      Look at the US in certain parts looking to or have rolled out this sort of tech based around surveillance cameras in the street etc which can then check a database of photos from government databases, police records, watch lists etc, this could easily be expanded to passport or drivers license photos.

      While we can be as smug as we like (and i often am about this) about keeping our IRL lives offline or not posting photos, being tagged by friends and others in photos in anything online its only a matter of time the governments roll this tech out everywhere.

      I mean look at the UN’s push to control the internet in a low key meeting in December.
      You think thats for the best? not a chance.

    • Audra Blue says:

      10:09am | 01/11/12

      I’d be very surprised if anyone could find me on the ‘net based on a photo.  I value my privacy and have no interest in being exposed to the world.  Hence, me hiding behind a nom de plume and putting up my opinions on sites like these.

      But I’ve always said if you’re on Facebook, you deserve every intrusion you get.

    • BVMkingmaker says:

      10:34am | 01/11/12

      Gavin In this day and age how many people can truly not be on the grid. Maybe not Facebook how about email of even posting on blogs or if you were ever in the paper, They can figure out who you are everyone and everyone is on the grid somewhere

    • Philosopher says:

      12:44pm | 01/11/12

      Audra Blue, I loved what you were wearing yesterday…although the red clashed somewhat with your lovely skin. *slithery sound*

    • Mahhrat says:

      06:38am | 01/11/12

      You know, I have a teenage daughter.  She has a Facebook account.  We are currently trying to figure out how to get me added because both our security settings are so high, we can’t even request to be each other’s friends.

      The thing with privacy is it’s becoming personalised.  We can’t expect the government to protect our privacy - they are not interested in doing it and never really have been - and we need to do it ourselves.

      The funny thing is, this sort of thing (and other techs like it) will slowly remove the ability for things like online trolling. Already, smart people can track you down easily enough using their systems - reddit again has proven its abilities to find douchebags multiple times.

      The only thing those in “power” don’t like about these things is that they personally don’t control it.  Welcome to the new age, baby.

    • Al says:

      07:39am | 01/11/12

      ‘The thing with privacy is it’s becoming personalised.’
      Gee, who would of gussed that privacy was actual a matter for the individual to keep private?
      The only time anyone else should become involved is if someone has published your private details without your permission.
      This is one reason why I fairly regularly Google (and other searches etc) my own name, to see what is actualy available. About all people can tell is that I may (yes MAY) have a Facebook Account (I actualy do), but can’t see any details etc. etc. (mostly I just don’t put them up at all).
      This technology won’t help anyone find you or stalk you if they can’t find any information.

    • acotrel says:

      08:12am | 01/11/12

      I’ve never liked forums where the participants hide behind user names anyway.  We have freedom of speech, and if you want to comment you should wear the consequences of being viewed as an outspoken dickhead.

    • Jack says:

      08:20am | 01/11/12

      Just FYI, your teenage daughter has two Facebook profiles. A real one, and the one where she adds her parents.

    • Al says:

      09:05am | 01/11/12

      acotrel - that comment is hillarious considering it is coming from a ‘participant who is hidding behind their user name anyway’.

    • Robin says:

      12:45pm | 01/11/12

      I have a Farcebook account, and a linked in profile plus a couple of other social sites.  The thing is, the date of birth is not real, the state I live in is not real, nor are any other details really except my name and the email account it is linked to is one I made specifically for each one.  Also the photo is of my dog, not my head.  The point is, I use FB and other sites to keep up with my friends, mostly ex-service mates, but even with high security settings see no reason why I should give my personal details to anyone, including the staff of FB.  So you can do things to limit your public exposure.

    • Philosopher says:

      01:05pm | 01/11/12

      thanks Robin. I felt like a right idiot, knocking on that door with a bunch of flowers and having to explain myself to the old man who answered. And after I summed up the courage to leave the house, as well! You’ve made me cross, my pretty one…

    • Robin says:

      02:42pm | 01/11/12

      Oh dear Philosopher, gonna have to ruin your day even more.  You know how my profile says female?  Well…........

    • Philosopher says:

      03:02pm | 01/11/12

      drat! *retreats further into shell*
      and to think I had our whole life mapped out…

    • K^2 says:

      04:31pm | 01/11/12

      Thats how Philosopher likes to roll anyway, Robin.

      Not only does he troll The Punch looking for people to astound with his superior humour, he also trolls facebook looking for guys that pose as girls online so he can rock up looking to party.

      He likes G.I.R.L.s (Guys In Real Life)

    • Philosopher says:

      04:43pm | 01/11/12

      looks like I ‘astounded’ the wrong person. Oh well, win some, lose some…

    • acotrel says:

      07:22am | 01/11/12

      I enjoyed the movie ‘The Truman Show’.  When I drive down through our town, it is just as though there is someone looking down controlling the traffic lights while watching me approach.

    • acotrel says:

      07:25am | 01/11/12

      ‘Like (almost) all technologies, F2D can be used both for good and for evil. This should not be ignored. We must at least consider regulating its use. ‘

      The philosopher Maxwell Smart is often on about this stuff.

    • stephen says:

      07:54am | 01/11/12

      ‘Was’ often on about this stuff.
      He got stuck to the cone of silence and got a stand at Gettys’ as a phoneringing toadstool - don’t you remember ?

    • Joan Bennett says:

      07:39am | 01/11/12

      Can you stalk someone like this if they are not on FB or other “social media”?  I’m guessing if someone wanted to stalk me, they would have to do it the old fashioned way and get into strife with my partner or me confronting them with a bat when they break in grin  Honestly, what is all this social media stuff?  It seems to be a lot of trouble and I don’t know a single person who is on any of them who is happy.  Clearly, interacting with people the traditional ways suits humans better than using this technology.
      I remember the first time I heard about FB, I thought, “Why would anyone want anything to do with it?”.  You put your life on the internet for ANYONE to see, including hackers who now have access to anything you do on your PC.  Are people’s intelligence levels shrinking with poor modern diets?  Is that why folks make really bad decisions these days?  Can anyone out there explain it to me?

    • egg says:

      09:57am | 01/11/12

      Oh Joan, so many assumptions… it’s cool, you don’t have to get it. And no, nobody can explain it to you. Just keep feeling superior and surely this “fad” will blow over one day.

    • Jack says:

      11:16am | 01/11/12

      Poor Joan: everyone else at the knitting club tells her that they aren’t on ‘the social networks’, when in reality they just voted her out.


    • PsychoHyena says:

      12:30pm | 01/11/12


      It all comes down to how you use social media, unfortunately humans, especially younger humans often fall into the needing to keep up with the latest fashion. I use Facebook on and off, it enables me to send a quick URL to a family member without much fuss, otherwise I can’t be bothered.

      Regarding your final paragraph though, there is information relating to you that is accessible via the internet regardless of whether you put it there or not. Have a driver’s licence? This can be accessed by hackers using the internet to gain access to the relevant Dept. Ever used Centrelink? Can also be accessed. ATO? Ditto. Even using The Punch you run the risk of someone obtaining your email address/ip address and using this to track down information about you.

      So no, social media isn’t the issue, the issue is the increasing level of computerised storage for the sake of convenience when there is insufficient safeguards in place to protect that information.

      A good similarity is your house, you might have amazing security and everything, but provided you have a method of entering and exiting your house it can be broken into.

    • Joe201 says:

      07:48am | 01/11/12

      It’ll just lead to a - probably small - group of people electing to “stay offline” by not having Facebook, Twitter or any other profile on a social media platform. For the vast majority though. I suspect tat these things which we view as “creepy” now, will be accepted and common place. “Hey, I found you on Facebook using F2D” will just become another variant of “I found you on Facebook”.

      It does make me think about the ad being run on TV now for “The New Normal”, how a gay couple are getting a surrogate mother to have their baby - and it makes the comparison with the “wholesome” Brady Bunch from the 60’s. Can you imagine the utter shock and horror if such a premise was put forward 60 years ago - as another woman having a child and giving it up for a gay couple?

      The times, they are a changing.

    • Cheezldoob. says:

      08:02am | 01/11/12

      This idea was put forward a few years ago in an anime “Eden of the East”.

      With Googles image search these things can be easily done. For those who don’t know, go to google search, choose images, then just drag and drop a picture into the search field, and it will return results which resemble that picture.

      The problem with the stalking idea is, what’s to stop the person to just following them home when they see them. Without this technology any person who would engage in stalking would just pick another convenient target. I’m getting tired of the Mrs Lovejoy refrain as an excuse against everything because of a very remote possibility, given we know the vast majority of kids are molested by people they know.

      Ideally there would be a bigger push to educate people across the board on what they give out an how business in general works, but business don’t want thinking consumers, just mindless drones.

      We live in an (allegedly) free market society. Business don’t develop what’s good for anyone, just what will make them money or what they think they can sell. How many people will remotely controlled/self driving mining trucks help? If the author is waiting for businesses to do the right/responsible thing, he had better believe in reincarnation - I’m sure in one of his lifetimes he will see it happen, more by coincidence than anything else..

    • Tanya says:

      09:08am | 01/11/12

      You can see the appeal of adopting an ‘alternate lifestyle.’ But they’d probably get you by satellite.

    • neo says:

      10:14am | 01/11/12

      Don’t use Facebook.

    • K^2 says:

      10:21am | 01/11/12

      Facial recognition is dodgey as hell.  I can guarantee anyone thats been thru an airport with a new passport is on facial recognition technology.  Reminds me of the movie “Minority report” where you walk down a street and all the adds are targetted at you directly based on what you clicked “like” on. 

      Cyberstalking goes beyond the “personal” this cyberstalking is being done by corporations, and the government is more than capable of doing it to you also, should they so wish, movies like minority report, enemy of the state, and the net are now more reality than ‘paranoid’ sci-fi. 

      Who watches the Watchmen?  This is a very VERY important question that has never been asked in relation to facial recognition, and who the info regarding us is being sold to!

    • Homer Simpson says:

      11:18am | 01/11/12

      I dunno, coast guard?

    • PsychoHyena says:

      01:09pm | 01/11/12

      I’m sure there were a few people who watched Watchmen, never managed to watch it while in the right frame of mood so it’s always been boring as hell.

    • Robin says:

      01:13pm | 01/11/12

      Well, there is that big fat fellow up at the South Pole.  He watches. He knows when you’ve been good or bad, naughty or nice.  So be good for goodness sake!

    • Philosopher says:

      01:59pm | 01/11/12

      careful Robin, K^2 hates humour.

      PS Watchmen was a very successful film… I thought a NERD like you would know that, K^2.

    • K^2 says:

      04:24pm | 01/11/12

      @Philosopher. Yawn, you ever think maybe you just aren’t funny?

      Obviously you didn’t watch the watchmen, or you would have known what I meant, but your superior humor obviously wins out for you most of the time over intellect right?

    • Utopia Boy says:

      04:48pm | 01/11/12

      It seems to me that protecting one’s privacy has become by necessity, an activity that requires direct and deliberate input from ourselves, and with sites like facebook, extremely time consuming. But that is they way it is.
      Here’s what I do:
      1. I love google Chrome, but it is little more than a data collection tool. I go “under the hood” and max out my privacy settings and opt out of everything I can. Adblocker plus running. Clear cookies after each use is an auto function.
      2. Facebook: Privacy settings set to max, not my real name, not my real date of birth, not my real photo, not my real location, not my real anything. Facebook is just for fun, not real life.
      3. This to me, is the most important thing, and it has nothing to do with conducting illegal activities or having anything to hide from the government: I use a multi functional VPN with two modes, OpenVPN and Open Web. Both connect to the internet in different ways, but primarily it shifts my IP to another country.  I run this at start up and never surf without it on.
      4. Given that the software in the above point is not infallible, and of course, if someone wanted to they could find my actual physical location, I also use TOR software which, to put it as simply as possible, breaks up data as it leaves my IP (now located in another country thanks to the VPN) and puts it back together as it reaches the next computer / server etc. TOR does make surfing slow, but if I value my privacy.
      4. Occasionally I will google myself (and my alter egos) just to see if any sites / apps have snared me. If I see something that shouldn’t be public, I go to the website, check the security settings for the site and if they are all maxed out, disengage from that site by closing my account / deleting program etc.

      Some of you out there might be a bit more sophisticated than me, so is there any other advice for laymen? Simply turning off the internet isn’t really an option these days, in any kind of permanent sense.


Facebook Recommendations

Read all about it

Punch live

Up to the minute Twitter chatter

Recent posts

The latest and greatest

The Punch is moving house

The Punch is moving house

Good morning Punchers. After four years of excellent fun and great conversation, this is the final post…

Will Pope Francis have the vision to tackle this?

Will Pope Francis have the vision to tackle this?

I have had some close calls, one that involved what looked to me like an AK47 pointed my way, followed…

Advocating risk management is not “victim blaming”

Advocating risk management is not “victim blaming”

In a world in which there are still people who subscribe to the vile notion that certain victims of sexual…

Nosebleed Section

choice ringside rantings

From: Hasbro, go straight to gaol, do not pass go

Tim says:

They should update other things in the game too. Instead of a get out of jail free card, they should have a Dodgy Lawyer card that not only gets you out of jail straight away but also gives you a fat payout in compensation for daring to arrest you in the first place. Instead of getting a hotel when you… [read more]

From: A guide to summer festivals especially if you wouldn’t go

Kel says:

If you want a festival for older people or for families alike, get amongst the respectable punters at Bluesfest. A truly amazing festival experience to be had of ALL AGES. And all the young "festivalgoers" usually write themselves off on the first night, only to never hear from them again the rest of… [read more]

Gentle jabs to the ribs

Superman needs saving

Superman needs saving

Can somebody please save Superman? He seems to be going through a bit of a crisis. Eighteen months ago,… Read more



Read all about it

Sign up to the free newsletter