Andreas Weigend (www.weigend.com)
The Social Data Revolution (SDR), INFO 290A-03
UC Berkeley, School of Information, Fall 2013
Class1– October 29, 2013
Andreas Weigend (www.weigend.com)
The Social Data Revolution, INFO 290A-3 (http://www.ischool.berkeley.edu/courses/i290a-sdr)
UC Berkeley, School of Information, Fall 2013 (http://ischool2013.wikispaces.com)
Class1
This transcript:
http://weigend.com/files/teaching/ischool/2013/weigend_ischool2013_transcript_1-1.doc
Corresponding audio files:
http://weigend.com/files/teaching/ischool/2013/weigend_ischool2013_audio_1-1.mp3
Containing folder of the whole series:
http://weigend.com/files/teaching/ischool/2013
Andreas: Welcome to Class 1 of the Social Data Revolution here at UC Berkeley in the fall of 2013. It's a short class. We have four three-hour sessions. That means we really have to concentrate, be really present, because there's lots of interesting stuff that's been happening in the Social Data Revolution. And I actually like, as a term here, the "social life of data" because that's really what we're talking about.
In order to get stuff done in this short amount of time we have in those twelve hours, there are a couple of things where we work together. The first one is there's a Whiteboard which I invite you to jot down notes, share notes in real time as you're in class. If you prefer to write it on a piece of paper, that's okay, too. I invite you not to take the notes on your own laptop, but if you take notes on the computer, then put them on that shared Whiteboard.
For that to be useful you need to reflect on what you learned in class. Given that this is a class about the world we live in, which is changing very fast, the way I structure this is after each class a quarter of the students, six of the twenty-four students here, are responsible for taking the raw material which you jotted down yourself, and others put down on the shared Whiteboard, and produce a page. Meaning a web page, and there are lots of examples from past classes for that.
I don't expect you to write a term paper. I don't expect programming work. There's also very limited reading. I expect you to think about the problems, to understand the tradeoffs with many of the things about social data, and the main deliverable you have is each of you, in a team of five or six, is responsible for delivering a very good page which in a nice structured outline mode describes what we talked about in a given class.
For that, I've created as a framework a wiki. The URL for that is ischool2013.wikispaces.com. Wikispaces is a small company in San Francisco, and I've tried many things including Google Docs, but I find for what we're doing in class and what I want to see as the end result, as a result of your processing things and then structuring them, so we can benefit from that, Wikispaces is the best platform I know.
If you replace the 2013 with 2012 it will show you last year's, which is an example. If you say Stanford 2011, Stanford 2010, etc., you see what the school on the other side of the Bay created as the outcome of the classes.
I've invited all of you to the email addresses I got from Bare Facts (ph.), to be a member of this wiki. The wiki is write by members, read by world. In order to actually edit you need to make sure your email is registered on it. If you haven't received the email that means you're not on the list I pulled this morning from Bare Facts. Not a problem, you need to check on this Google spreadsheet that your information is correct. Go to the Wikispaces and click on "request access to this wiki" and I'll just grant access to everybody who needs access who doesn't have it yet.
What I'm doing today is I want to walk you through a few verticals, through part of our society, part of our business, part of our world. And show you in each example how I think social data has dramatically changed or is in the process of dramatically changing how those parts of our world work.
Examples are as deep as the future of work. Let me start with that example. You might not know but in the olden days people showed up for some interview at a company. Maybe they had a printed piece of paper called a resume. Besides that, the company knew very little. There was no social media presence. For instance, people didn't answer questions on Quora or had a LinkedIn profile where people could actually check the connections they have with others.
They spend a day in that company, talking to people, making up stories. And in many cases stories that are well rehearsed through coaches. Then based on that performance, people get a life-long employment job. That's how this used to work. Probably doesn't sound familiar to anybody here now.
The world I see we live in is that we leave so many digital breadcrumbs of what we do. For instance, one of the startups I'm working with just hired a developer because that person answered a question on Quora. They thought that's a good answer, and it turned out he was willing to work with them. Forever? Probably not, but certainly as long as it's fun for him and worth it for the startup to have him. Those digital breadcrumbs we create are that change, in this example, of the future of work.
Yesterday I was at oDesk. oDesk is a company on the other side of the Bay in Redwood City, where worldwide, for all kinds of work, you find matches between people who are willing to solve a task for somebody who is willing to pay for having that task solved. For example, the person who is going to create the transcripts of our classes, I found her years ago through oDesk. I've never actually in person. She lives in Israel and it is one of those future of work examples where she, when she has time, produces a transcript. I think honestly she knows everything I've said because I've said the same thing many times in my classes J. And then she ships it back.
Very different from having somebody sit in an office and be there when you need them. oDesk is thinking deeply about the pricing of work. The model many of us have that we're used to being paid an hourly rate makes no sense anymore. As you know if you're a programmer, a really good programmer managed to get an hour done what a really bad one doesn't get done in a whole week. So we have many examples that come from a world -- this is important now -- from a world where things were not observable, where data weren't created.
That led to certain models of how we think the world works. How for instance employment works. oDesk takes screenshots at random times so if you pretend you're doing work you better not be surfing the web, or watching porn, or whatever else you might be doing because that will be very difficult to claim that that was actually what you were doing in that time for that employer.
That is an example of how the data we create, and in this case that we knowingly and willingly agree to having screenshots taken, can provide transparency. We do those things, we do well more. And when we realize actually what we're sometimes wasting our time on, we do those things we don't do well less often, or find other people who can do it for us.
I'm a physicist by training. I did my undergrad in Germany. I worked at CERN in Geneva. What CERN does is it understands how particles interact. You might have heard about the Higgs Boson last year, so that's an example that physical phenomena became observable in the last century.
I then came to the States and between universities I had a choice of, I decided against Berkeley and went to Stanford. I started my PhD there at SLAC which is the Stanford National Accelerator Center which is the place where they produced large amounts of data, because I was curious; how can you engineer experiments by having those particles interact with each other so we can learn something about nature.
I think it's fair to say the last century really was the century of physical sciences. Why? Because things became observable. Think about X-rays. X-rays were discovered in the late 1800s. Not that long ago.
Or think FMRI. Last quarter I taught a class at Stanford with Brian Knutson, where we're trying to look into how people make connections. Brian is a professor in psychology. He shoves people into FMRI machines, big, expensive magnets where he measures with a one-second, one-millimeter resolution what happens in their brain, which areas are active in their brain as he shows them stimuli.
Like with a microscope, he measures how people make decisions. By the way, the news is pretty bad. It turns out that he can predict from what he sees in that first few seconds of you being shown a stimuli what you're going to do months and years later. He does experiments by putting people into magnets, microscope.
I look at the behavior of people by looking at the data they create anyway. So-called implicit data. And we see lots of examples of implicit data. Then sometimes I combine them with explicit data when we ask people questions. For example, I was the chief scientist at Amazon.com. In one of the board meetings, John Doerr, a venture capitalist from Kleiner Perkins wanted to know why do people come to Amazon.
He thought maybe we'll do some datat analysis, look at all those clicks people leave, maybe the reviews people write. But I thought let's combine data people are giving us in response to the question, why did you come to Amazon today, with the implicit data they leave.
We showed 100,000 people a popup window asking them that question. Then we wrote, an iterative process with the data, a program that manually tried to categorize the answers. Then for each of them, we looked at the click stream. What did they enter as a search term, what did they click at, do they buy something at the end of the session? That is an example of how we ask questions now in a very different way because we can see the interaction between people and the world we create, in this case the online world of an Amazon.com.
I think that just like the last century was a century where we had amazing breakthroughs in the physical sciences because physical properties and interactions became observable; this century will be the century of the social sciences because social interactions between people are becoming observable.
Think about Facebook. Facebook was a brilliant platform that instruments the interactions between people. It is not just like a phone company where you have a binary signal that you're giving me a call and I'm answering the call. Think about the multitude of how you can express your relationship with the other person. In other words, think about the multiple ways Facebook knows what kind of relationship you have.
If whenever you post a picture, the first thing that happens is I like it, then you know I'm checking you out. On the other hand, if you message me and I never respond, then maybe I'm not interested. When we think about a social graph, it's not just a graph where the connections either exist or don’t. It's a graph where the connections have color, have many dimensions to them; where the connections usually are asymmetrical.
It's a graph we can analyze. Why? What would you do with it? We want to show people stuff which is of interest to them because in this world of big data there's so much stuff out there. That means we need to use the data to actually filter, to show relevant results, to try to provide a user experience so people will come back. That's not done by asking them to do things explicitly. It's done by understanding what they do and by setting up an ecosystem so they, by truthfully doing things they believe in, will see a better future themselves, will see more interesting results.
The point I made was the last century observations in physics became possible, breakthroughs in science. This century observations in social interactions between people have become observable. We're expecting amazing breakthroughs in social science.
If we roll the clock forward by seven years, to the year 2020, we'll probably all be wearing glasses that tell us the emotional state of the other person. It will tell me who of you is really interested right now and who of you are thinking about what to do for dinner.
We're pretty far already. Yesterday I was at a VC in the afternoon with Jerry Yang who was the Yahoo founder. He supports some startups with the money he made. There's one interesting one which has devices that pretty well know based on your skin galvanization and other measures how excited you are. They're not cheap yet. Jerry's giving me one of those watches of his, $1,500. But I wear it and I'm curious when I'm excited what can I learn about, in this case, just my skin galvanization.
Think about it. If you were all wearing such devices, I could reach how each of you were doing. Some are bored because I'm too slow. Others are lost because I'm too fast. I could actually react very well to that and explain things. If it's too diverse in the group, then maybe split you up. Those of you who are already totally getting it, could explain to those who are not quite there yet.
Think about what that effect would be on how education works. From this setting which hasn't changed for the last thousand years where some dude stands in the front and other people sit in the back, and it's sort of synchronist here. Think about how getting more feedback actually would influence the way people learn.
We talked about the future of work. We touched upon the future of education. Let's look at another example, the future of commerce. For that example I wanted to actually spend a few minutes telling you how Amazon does recommendations so we at least know what the past of commerce was.
Maybe we should go back 100 years and think what data did the storekeeper have. He knew how much money was in the till. Maybe he kind of knew what his inventory was. Then of course if it's a little town, he knew who the people were that were coming and buying stuff.