GroupScope: A Multimodal System for the Study of Large Group Dynamics
Interested in collective action, tracking emergence service providers (live world tracking)...want to track large groups (hundreds...)
GroupScope: an envionment that will develop technologies to understand social interactions with a fidelity that enables breakthrough social research.
Claims that he hopes the computer will do the drudgery and leave him to do the fun stuff; however, as the group of us work on coding the agenda...that is still requiring some of the drudgery (and the agenda is a short text!)
Learning about HathiTrust (large text archive/corpus)
Reminders that the point is to understand what goes on under the hood of serious image analysis applications so we have some sense of what/how/why to have conversations with CS folks in the future (we're not going to be coding this stuff ourselves).
What can you do?
Matching: match an object in two images based on similar features.
Tracking: Follow an object in a video by following its features.
Object Recognition: Find objects based on known features it will posses.
Segmentation: Break an image up into more meaningful regions based on the seen features.
We have large collections of image data...what/how do we do with it?
1. A high level understanding of what Computer Vision is and how we might use it.
a. A sense of what is currently possible.
b. A sense of how these things break
c. A sense of what might be possible
d. A sesne of what is pure science fiction
e. The looming opportunity in "Big Data"
2. A little bit of hands on experience.
Computer Vision: What information can you get from a digital image? High level and low level (and mid). What kind of scene? Are there cars? Where are the cars? Is it day or night? What is the ground made of?
What computer sees: raster Images...pixels, matrix of numbers.
How/why is this hard?
Image created: a scene, light bounces off scene into a sensor (eye/camera lens). Variables:
Light/s: position, strength, geometry, color (Shows example of the light source of an image can change how the machine understands a color.)
Started showing a picture of a smell sensory device/experiment (what smell versus what see & taste).
Alan is interested in interact/interface with computers & the world.
Talking about "Caves": fully immersive virtual reality spaces.
2 big tasks:
decide what you want to do with it
Caves are only as good as the software.
If the person's body is not encoded it is diconcerting; discussion of easily making people trip/fall without ever touching them.
Lot's of Twitter backchatter about the ethics of VR (esp. with the high cost of Caves and access).
Alan is now talking about more accessible VR, things like a "fish tank" VR.
Alan's Definition of VR: a medium composed of interactive computer simulations that sense the participant's position and replace or augment the feedback to one or more senses--giving the feeling of being immersed or being present in the simulation.
Sensors: to determine pose and info about the world
Smartphone & Tables have them all!!!
Fiducial Symbols/Markers: something you put in the world that makes it easy for the camera to recognize.
Natural Feature Tracking (NFT): Faces, Skylines, Natural Features, QCAR-Qualcomm (Qualcomm example allows for interaction with "buttons" that changed the color of the augmented object.)
*representing information in a way that is maximally benefitial to the intended audience
a. information visualization
b. scientific visualization
c. other specific areas like Business Visualization, Humanities Visualization, etc.
*provides NEW insight OR a NEW way to communicate
1987 NSF Panel Initiative-Formal Definition
"Visualization is a method of computing. It transforms the symbolic into the geometric, enabling researchers to observe their simulations and computations. Visualization offers a method for seeing the unseen. It enriches the process of scientifice discovery and ofsters profound and unexpected insights."
A. Craig's interpretation: it's a method of doing something
Richard Hamming: "The purpose of computing is insight, not numbers."
Goal of visualization: leverage existing scientific methods by providing new scientific insight through visual methods.
Visualization: The choice of the appropriate representation of what we are trying to do.
Interpolation: Generating new data based on a given numbers.
Inveractive vs. Batch Visualization
Interactive: allows the ability to control in real-time; limits the amount of data; useful for analysis & exploration (more likely to lead to new discovery)
Batch: High-Quality, complex representation; no control in real time; useful for presentation, communication, high complexity (focused on presentation)
Data, filter, map to geometry, viewing attributes, render, display, record, loop to appropriate step...
Neighborhoods of Pixels...so how do we do something with this!
1. Interest points: Points or collections of points that are somehow relevant to indentifying the contents of an image.
2. Subset of the image: Far less data than the total number of pixels.
3. Repeatability: A feature detector should be likely to detect the feature in a scene under a variety of lighting, orientation, and other variable conditions.
Changes in intensity: albedo, orientation, distance
McHenry uses MATLAB to demo how to read/manipulate/interpret digital images.
Convolution: the concept of the function scanning the whole image
Continuous vs. Discrete...real world, smooth; image is static, discrete, pixels! So in MATLAB compare vector coordinates of given contrast (like vertical distinction) to vector coordinates from an image to mark what you are looking for (we're talking about borders of objects in images). To smooth out noise, average the difference of the surrounding pixals and make the center pixal that new average.