Whole body interaction

Introduction
What is “Whole Body Interaction”: The integrated capture and processing of human signals from physical to generate feedback for interaction in a digital environment

Challenges
1.Design for Whole Body Interaction
2.Engineering of Interaction
3.Research Philosophy

1.Design for whole body interaction: In the past we use our bodies to do things such as walking, lifting. Nowadays technology are evolved, we can use our bodies to interact with the electronic device.
Start with the premise that all aspects of the human user are involved in interaction and elements removed are not required. Next, specify how elements are integrated. The interaction design with the individual elements will be expanded. The design process would be iterative for discovering more information about the users. For the example of
- Multi-user air guitar: We are able to play(control) guitar by standing in front of the camera without using the real guitar. User can use floor pads to produce effects by changing the pressure of the pads. Optical flow will measure the speed of hands across the string to produce musical notes.
12.Engineering of Interaction For the example of
- Wiimote from Nintendo or the Microsoft Kinect to use in non-gaming domains.we need at least four points of reference on the body whether from sensors or cameras. We also require the processing and feedback of signals to be in real time.

3.Research Philosophy: Human Computer Interaction is a multidisciplinary subject. Much of the work in Whole Body Interaction takes place in cross-disciplinary contexts.Whole Body Interaction is in one sense a reaction to that fragmentation and attempts to bring us back to the user as central in our design and research thinking. We need to raise questions about our research philosophy and methods.
- Motion capture: the process of recording the movement of objects or people. It is used in military, entertainment, sports, and medical applications, and for validation of computer vision and robotics. In filmmaking and video game development, it refers to recording actions of human actors, and using that information to animate digital character models in 2D or 3D computer animation.

00:0000:00
- Physical interaction

Whole Body Interaction in Abstract Domains
- Conceptual Metaphor Theory posits that human capabilities for dealing with abstract concepts are always grounded in a small core of universal sensory-motor abilities, together with their associated low-level inferencing capabilities. Conceptual metaphors can be applied in two different ways: as grounding metaphors, and in conceptual blends
Grounding metaphors enable the simplest abstractions to be grounded directly in physical experience
Conceptual blending works by allowing two or more conceptual metaphors to be used to create new expanded or composite metaphors by processes of composition, completion and elaboration

- A Complex Abstract Domain: Tonal Harmony
Tonal Harmony concerns the organisation of multiple simultaneous pitch sources, for example, two or more singers, or instrumentalists, playing independent but coordinated melodic lines. To see how this relates to whole body interaction and conceptual metaphor, it will help to briefly consider two related theoretical perspectives, namely Sensory Motor Contingency Theory and Dalcroze Eurhythmics
Sensory Motor Contingency Theory: Sensory motor contingency theory shows that in order to learn to organize. And respond appropriately to sensory input in a new domain or context in which it is. Important that the actions of one motor with the power to affect . Relationships observed in domains in that situation. There are a variety of specific types of reviews and . Experience is the lack of ability to be treated to failure of development principles. Trained singer And play audio out only once. May be able to work together musically . May be due to lack of experience in controlling its use.
Lessons from Dalcroze Eurhythmics: Dalcroze Eurhythmics, applies to musical rhythm rather than harmony, but shares key issues with tonal harmony. In order to encourage competency in enacting rhythms, particularly ones involving multiple streams, Dalcroze invented a system of rhythmic ‘gymnastics’ or ‘eurhythmics’. We propose that these insights from Dalcroze Eurhythmics can be applied fairly straightforwardly in principle to tonal harmony.

- Harmony Space – A System for Exploring Tonal Harmony
Harmony Space is an interactive digital music representation system designed to give beginners and experts insight into a wide range of musical tasks ranging from performance and analysis to composition. When combined with whole-body interaction to create the Song Walker version , Harmony Space allows users to enact complex harmonic phenomena physically via mappings between (a) bodily movement and (b) conceptual metaphors and blends for musicalabstractions.
- Conceptual Metaphors and Blends in Harmony Space
Harmony Space exploits a set of conceptual metaphors that link concepts in tonal harmony to spatial concepts.  Harmony Space in a combined technical, and cultural process in three stages, as follows.
1.Harmony Space introduces and embodies a set of metaphorical blends in whichthe basic conceptual metaphor of pitch interacts with itself in a series of layers.
2.The interactive digital nature of the tool facilitates the visual plotting of diverse music from the repertoire in spatial terms. This process affords reflection by interested users and music analysts on the emerging patterns and their musical meaning.
3.From the above technical and cultural process, new ways of discussing the technicalities of harmonic phenomena have emerged using a spatial, movement-based vocabulary as an alternative and complement to symbolic technical terms.
2
Whole Body Large Display Interfaces for Users and Designers
Ever since the introduction of the mouse and the WIMP (windows, icons, menus,pointers) model of human computer interaction, users have been required to use computers in the same consistent manner, regardless of the task at hand. Work and interaction in the physical world is much more flexible, fluid, and dynamic than work that is constrained by the limitations of a traditional desktop computing system.
Large display devices offer benefits for interaction. The form factors of such displays mimic the properties of large physical work surfaces, such as whiteboards and tables. As with whiteboards and tables, large displays can foster collaboration and support tasks such as brainstorming and casual data reference.
From a designer’s viewpoint, body-centered interaction eases development, and helps the designer keep the user central to the design process. We examine both of these viewpoints, with a particular emphasis on interaction techniques for large displays.

- Large Surfaces and Large Displays
Large physical surfaces play an important role in supporting everyday tasks. As such, they ought to be a critical component of the whole workspace “toolbox”. Whiteboards and tack-boards are widespread in homes and offices, and play quite different roles from desks, due to their vertical orientation.
4              3

- Body-Centered Interaction
There is a rich history of work investigating the relationships between human bodies and virtual interaction techniques, both within and outside of the research community. Artists have often been at the forefront of this exploration. A very early example of this is the VIDEOPLACE project by Krueger et al., which made use of a shadow embodiment of the user to mediate interaction.
5Why Body-Centered Interaction
•Display surfaces scattered about the room.
•Easy reach of the hands instead mouse or laser pointers.

- Interaction Spaces: It is believed that interaction in personal and peripersonal space is more optimized than interaction in extrapersonal space, because most of our real world interaction experience is inside the bounds of peripersonal space.
Personal space: Occupied by the body
Peripersonal space: Region within easy reach of the hands
Extrapersonal space: Whatever lies beyond peripersonal space

- Social Conventions
One important aspect of social interaction is the notion of private space, as described by Felipe and Sommer. Private space is the region around a person’s body outside of which they attempt to keep other people during normal interaction. As described by Sundstrom and Altman, private space is more complex than simple private/non-private, and changes fluidly depending on the situation.
For example, a system could use proximity information to determine when different users transition between close collaboration and independent work.

- Example Interaction Techniques
Based on our knowledge of interaction spaces, and the scale of interaction for large displays, we have developed techniques in which users interact with the display through a virtual shadow proxy.
Body-based tools:  allows users to select interaction modes by reaching to predefined body locations. This approach relies on the power of proprioception, making mode selection a personal operation, and thus avoiding the need for public tool bars and menus that clutter the workspace.
Sharing protocols: in which real-world physical actions result in the copying of virtual data between users’ personal data stores. The protocols involve users directly handing virtual content to one another.This incorporates common social conventions directly into the process of sharing digital data.

- Body-Centered APIs
Application programming interfaces (APIs) allow developers to create software applications without requiring knowledge of the low-level details. It follows then that body-centric development should be supported through the use of a body-centric API, or BAPI. A properly designed BAPI allows developers to query the state of users, including the locations and limb poses of individual users, and relationship between users and devices, displays, and each other using a single model of the entire scene.

Capacitive Sensors for Whole Body Interaction
Capacitive sensor: measure changes in capacitance between a sensor antenna and its surrounding. Capacitive sensors offer a cheap, robust, and flexible way of prototyping and implementing sensor systems for Whole Body Interaction.
Three different capacitive sensing techniques: loading mode, transmit mode, and shunt mode.

- Related Work
Probably the first use of capacitive sensors for primitive Whole Body Interaction was in 1907. German physician Max Cremer put a living frog between two capacitor plates. By measuring capacitance changes he could observe the frog’s heartbeat.
In 1919 Leonid Termen presented the Theremin, a musical instrument using two capacitive sensing antennas for controlling pitch and volume of a tone . The Theremin is played by hand movement near each antenna. Termen used the same sensing principle for tracking body movement Recently, capacitive sensors have been used in a variety of human–computer interfaces. Rekimoto determined hand poses by capturing the wrist shape using a wrist band containing capacitive sensors.
Taylor and Bove equipped a baseball with a capacitive sensor matrix which allows to determine how the ball is being held.
Valtonen et al. track users by equipping floor tiles with capacitive sensors. By embedding sensors into furniture, one can unobtrusively capture everyday actions.

- Game Controllers Built from Capacitive Sensors
The games’ concepts and graphics were kept very simple in order to reduce confounding variables and provide a coherent feel across the three games.
MoveBall:  A computer screen showed a 2D, top-view arena with a randomly placed ball and a randomly placed hole. By approaching one of the sensors with the hand, the user could move the ball in one of the four directions. In each round it was counted how many times the user could move the ball into the hole within 1 min.
HitBall: The user stood on a large sensor plate. Two antennas were placed at arms length to his/her left and right. A red circle appeared randomly at the top, left, or right edge of the screen. Depending on its position the user had to reach out towards the left or right antenna, or jump. In each round it was counted how many times the user “hit” the correct target.
WalkBall: The goal was the same as with MoveBall. The user could move the ball forward by walking in place. the user could change the ball’s direction. In each round it was counted how many times the user could move the ball into the hole within 1 min.

Conclusion
Due to some unique properties, capacitive sensors are especially suited for Whole Body Interfaces, especially for game controllers: Robust hardware, Flexible placement and layout, Friendly but challenging behavior.

Group Member
Nuttida 54270613
Praewpun 54270662

Big project 2

During the beginning of work I come for work but we have not to divide responsibilities clearly so in about coding, I don’t do it only by myself we help each other. Although I do not coding but at least I know each sensor work because I come and see when my friend coding on it and help them to research on how it work. Other thing I help my friend to check the circuit by using multimeter, check circuit and make sure that it can work. I help my friend to stripline(ปลอกสายไฟ) because wire that we get it does not strip out the copper so my friend can use to put in circuit and I should do it carefully. And I had a chance to see them fitting at the theater, we see some problem such as untidy wires and one of them fitting cloth but I still have accident the battery is explore but it’s safe because volt is only 9-12v. Then we come back and fix that problem. So this happen before we have allocate the duties. In that day we got to the partition function is completed. It is a very good thing, because I intend to do my part as well. And I start to sew a wire on cloth on that day, the first cloth that I sew is vest. We use needles and ligament for sewing the clothes. And I have 2 points that I think it’s very hard for sewing, first is shirt collar because it’s very thick, it difficult to prick and it makes my fingers hurt. And second point is after we sewing the wire into the shirt finished then it will have the remaining wires, I have to sew it into the inside of the shirt. Oh my god I have to burrow into the inside of the cloth to sew it. It’s difficult but fun. And after I finished my some duties so I will help on other people such as I help on my senior to solder to make a battery of shoes. I bring wires and solder with battery for using with shoes. And for battery casing we built it by 3D printing in office that very cool! I have sewing the clothes such as vest, skirt, pants, suit and dresses. During the time I was working the afternoon until late at night because the morning was not a lot of people to work. They will come around 3pm onwards for working until late at all.

 I have learned a lot on this project and this subject. The first, I have more about how to sew the wire on clothes which I have never done it before, know how to use knowledge that we learn from circuit and programming to apply in real work and know more about different sensor that we use with LED. The second, I learn how to work as a team, how to manage my time to do work that should be finish on time for to be completed the work as soon as possible. The last is if it was not for this course I would not have the opportunity to study in various places. To know that knowledge that we have learned it can be used to advantage in making interactive technology. And I see this interactive technology not only in museum, it also in various exhibition. I have chance to visit theatre of Kun’Pichet. Most people don’t know who he is because almost showing in other country. And I’m lucky to have responsibility to work with them that they are going to use in the next show in Singapore.

BIG project for Chang Theater

For our project that we do for Chang Theater. We create clothes with LED light and pole with LED, I will give the detail for this project.

In 9/05/2557 is the first day that we're come to meeting and our senior let us to do the work. We divide into 2 group : programming and hardware.Before we working our senior talk with P'Bo the designer about clothing style. I choose programming and in first day, we can program on auduino a little bit for test led can work.But here is an overview of duties. In other day, we have divided into several duties and I am  responsible for wiring of clothing.

For programming with Auduino the job that we have to do consist of :
1. Make it can on/off every Led through Wireless and have simply interface for using. In the beginning we try to use Xbee. We try to connect xbee with computer and check it, it's not work because we doesn't use xbee long time and it's very complicate. So we change into NF24 instead Xbee because it's more simply and cheaper.

This picture we before we use NFR, we try to use Xbee first but it's not work.

2. Control LED strip that we install on pole. In this show, when actor touch the pole the light will turn on and move around the pole. Luminescence of the LED is depend on how strong touch with the pole. In this we use Piezo Vibration sensor is used to aid in the vibration force.

3. Turn on/off LED via button. We put the button with the costume, it make our job more convineince.

4. Make all Led lighting by the volume of the music. In this part we use microphone for receive sound if music is lound so the led will have more light.

This is overview of our project, I will give in the detail later!!

Study at โรงละครช้าง

30/04/2014

Today Kru'Pichet Klunchun tell us about meaning of LED light. He teach us that the role of artist is make light have the meaning and give imagination to all the people. Like when you watch on the stage, when the light is turn on or off all of this has the meaning for audience. Technology in theatre all the light is complex in audience's feeling. The light acts resulting one-dimentional, like we use one light to focus on the only one actor that is we can see the meaning of light. If we want to show performance that without actor, we can use only the light is also can tell the story too ,and the light can make audience to have feeling or emotion with the show. Light can be made to feel our fear, excited, sadness, funny and etc.
Another job that important is lighting designer. It is meaningful process then use light to act define it. Lighting designer will use the correct lighting to convey meaning.
In this work, three people are working together:
1.director
2.lighting designer
3.music director
They have the power to decide as well. In Thailand we don't have lighting designer. This has a lot of impact, the spirit of the people in a society hardened, social problem. Kru'Pichet told us he use lighting designer of Japan. On the state he told us, they have a problem in performance dance :
1. Energy - audience can't see that dancer have energy.
2. Can't feel all muscles of the dancer while working.
3. How to make audience to feel tired and breath of dancers.

----------------

In this show thay devide into 3 part that Kru'Pichet need us to develop for them

1. Shirt 

    The idea of this shirt : 1.) transparent shirt - it about sexual, to know the real of people

                                    2.) light - to show military , to show a powerp

    Meaning - 1.) we can convey the emotions and feelings of the person through the light without movement or minimal.

                    2.) Emotion or feeling of human, we have our limits. Hence the need for technical help convey the mood is intense.

2. Pole 

    The idea : How do we convey the pain. When hit the pole and people can't hear the sound so it can make audience can't understand what actor want to say.

     Meaning - Talking about sexual men because in art can not communicate directly.

3. Water bag

    The idea : they want to build another world. For people to escape from the world of reality into another world.

So the next job that we will do is develop the bodysuilt with have led light, develop the pole with led light and etc. I will show the work in next time SEE YOU!!