Q: You were part of the Xerox PARC team that developed the world’s first GUI. Today you are exploring alternatives to existing GUIs that are not only more accessible people of traditional cultures are also inspired by those cultures.What motivated the journey how has it been so far?

A: Computers are just an expression of the world-view in which they exist.They are currently situated in information theory and western literacy-based models. I want an expression of alternative models for those cultures which work with nature, and which do not just believe in intellectual and rationalistic determinism. Designing computers for the ‘rest’ of the world then requires rethinking of user interfaces and paradigms, not only in software but also in hardware and product design. If my vision is realised, I hope that the process of engaging traditional cultures will generate income for those craftsmen and artists who are about to get marginalised. In the ideal world, people interact with their objects of meaning, not computers. Computers can be integrated in objects of meaning.

Q: Computers becoming integrated in objects of meaning?

A: Yes. What do we do today? Drive to work, walk into the office, and then the computer becomes the focus for the next eight hours or more. In workplaces as well as in 21st century homes, not only computers, but hardware in general, seems to command a lot of attention. A simple example is how television sets have become the centre of attraction in most living rooms. Now, imagine a situation where computers become ubiquitous and blend into the environment. The columns and archways in your offices, the windows in your car—all become user interfaces. Screens need not always be physical. How about projections on the wall, or literal desktop computing, where your desktop itself becomes the computer for you? This way, you would ‘take the work out of work’ because you do not focus on a ‘computer’ throughout the day. Your natural surrounding itself is the UI. But a lot of technological advancements need to come about before we can actually implement this. The UI in mobile computing devices today is still in the Stone Age, compared to this vision. Hardware and software need to evolve. It requires a radical design change and thinking beyond black-and-beige designs. Gradually, concrete physical displays will disappear and blend into the traditional and cultural surroundings. Computers will become instruments of cultural expression rather than replacing them.

Q: Can you explain the concepts of culturally rooted computing and culture-conscious product design?

A: Culturally rooted design basically explores how technology can become a vehicle for preserving cultural identity, not replacing it. If objects that people interact with for more than eight hours a day (such as cell phones and computers) can provide valuable remembrances -such as in the case of Gandhi, it could remind one of non-violence and the equality of religions—then I believe products can also have a spiritual function. For example, imagine a computer in the form of a shawl. When you wear the shawl it starts uttering mantras or japas. Note that I am not hiding the computer in the fabric but making the fabric itself the computer. This is possible. When we did a multimedia exhibition on the city of Banares, called “The Crossing”, we developed wearable jackets, which acted like say, an iPod. The gold thread of Banarasi fabric is conductive. So, it is possible to weave the computing elements into the fabric itself. The motifs embroidered in the fabric act as ‘hot spots’ and trigger multimedia resentations when the wearer touches them. All this is therefore possible—it just needs the imagination of more companies prepared to research and develop products that integrate with the rich resources already found in our culture.One of your earliest projects was the Chinese Temple Editor (1985-86) where you developed one of the first keyboard-less, gesturebased interfaces for designers to use graphical gestures as a means to interact with, index and retrieve records of the design process.

Q: What inspired the project?

A: The Chinese Temple Editor was conceived when the world of computing displays was ‘green text-based screens’. There was a culture of written text being the centre of the world, and because of my artistic background, I questioned it. Look at it this way—even today it is possible to search easily for text, but not so for images, sound or video. While I can feed in text into a search engine and say, “Show me similar texts,” and can obtain results easily, I can’t feed the image of a Chinese temple’s roof and say, “Show me other temples with similar roofs.” This is a challenge that has not been met even today, but we took a step towards it in 1985, by developing a parametric way of looking at Chinese temples in terms of architecture, roof and column structures and objects. Instead of looking at a temple’s image as a bitmap file, look at it as vocabulary elements (much like how you create an image in a graphics editor, as different objects on layers)—the pillars, the roof,
the windows, each as a separate object. It then becomes easier to edit the image or search for similar objects and so on. While such a concept might be easier to comprehend today, when we did this way back in the 1980s, UIs were still all about green text on a black screen. As for gesture-based computing, we found that it is easier for people to express themselves through gestures than through keystrokes. So, using a combination of camera inputs and the mouse, we allowed the user to interact with the computer through gestures. For example, the user could indicate the shape of the roof with her hand, or she could move her hand up and down to specify the height of the roof, “A tad up, a little down,” and so on, which is easier to express through gestures. So, gestures here not only provided the command but also the dimensions of the temples. Many cultures believe that the mind is in the hand; if the mind is in the hand then the interfaces need to be richer and
promote kinaesthetic interaction and internal well being. That is the key to gesture-based computing.

Q: In 1993-94, you came up with Hyper paper, a paper-multimedia cultural learning tool, where pointing and clicking on ‘hot spots’ printed on the paper pages of a book triggered a playback of short multimedia recordings in a computer. No mouse-clicks, no key-taps, but a direct user-papercomputer interface. Can you tell us more about Hyper paper?

A: In the 1990s we were experiencing the digital revolution and the paperless office, and the is appearance of paper was a pretty debated topic. But we realised that paper was a rich medium, not just in the text it represented and conveyed, but in the way paper as a display could be folded, not to mention the whole art of manuscripts, etc that evolved around the paper genre of book design. Also, paper was portable—you could lie in bed and read it. We wanted to combine the benefits of the computing media as well as the tactile dimensions of touch and portability of paper in Hyper paper. As to how Hyper paper works—well, there was a pen that the reader could move over the book. The pen was connected to the computer using a wire (remember there were no well-developed wireless technologies at that time). The pen had a camera in it, which could trace the ‘glyphs’ in the paper, encode them and communicate them to the computer. When the computer detected certain known patterns transmitted by the pen (when the pen moved over ‘hot spots’ ), the playback of a multimedia presentation was triggered. The video screen for display was inbuilt in the book cover. This logic can revolutionise the way we work. Imagine, if a pen could encode the movement of your hand and detect patterns in it. Then you could just write in thin air and have the message sent to somebody by e-mail. No more irksome typing with your thumb on mobile devices. Of course, this is just a possibility. Perhaps one day it will become reality. The key is in understanding that integrating technology with cultures, traditions and the way we live is all about identifying the patterns and social meanings of how we act. For example, if we could analyse a voice recording, and we identify a considerable pause just before expression, then we could guess that the person was thinking before expressing an idea. Similarly, if we detect shrill notes, we could guess that the person is angry or excited. The idea of a pen acting as an encoder and detecting the patterns in your writing is similar. It is through such abstract notions and a deeper look at cognition that echnology will gradually weave itself with social meaning.

Q: The Crossing, a project that you spearheaded in association with Xerox PARC in 2002, took viewers on a virtual journey through Kashi. It had seemingly magical eggs with high-touch user interfaces that interactively educated the holders about mythology, a high-tech cyclerickshaw took the riders on a tour of Kashi depending on the directions they chose, mendicants could be asked for directions on the way, and a variety of novel information access and delivery methods. You made the secrets of Kashi available even to the average Indian who did not know the ‘language’ of mouse-clicks and keytaps. The project was one of the earliest to implement wearablecomputing, body-friendly handheld interfaces, 3D presentations, and
tactile computing. Can you tell us about the absolute and almost seamless interface between technology and tradition exhibited in the project?

A: The Crossing was a major milestone in interaction design, not just because of tactile computing but also at the level of technology becoming an instrument of cultural identity. It serves as a benchmark for the interface experience. A panorama of tactile interface actions, such as tilting, touching, turning and sliding was explored in a variety of information browsers. The project aimed at showing that computing could be ubiquitous. Computers embedded in Shiv Lingas would present content to the spectators. To them, it looked like a linga, but it was actually a computer. Images of dancers and musicians would emerge and talk to people. Touching symbols could trigger a multimedia presentation. These are tactile and ulturefriendly interfaces which even conservative people steeped in tradition have been comfortable with. At the level of bringing computing to the rural areas, villagers won’t want to interact with keyboards, right? This is because the idea of files and folders comes from a western work practice. But they will be comfortable touching traditional symbols such as a charkha.
These could very well act as the UI. So, tactile interfaces provided the solution to develop culture-friendly interfaces that everybody would be comfortable with.It was our way of capturing the culture and tradition of Banaras or Kashi, which was once upon a time a renowned centre of education and arts—the oldest living city, a place so transformative that people went there never to come back. It was believed to be a crossing point from the worldly to the spiritual world. We saw how we could juxtapose such a spiritual environment with modern life, which is at odds with traditional society’s notions of spiritual connection. And technology helped bridge the two. It also proved that India has an immense resource of aesthetics that can shape modern tools. As I said earlier, cultural reflectivity addresses the question of personalising the forms of computing, so that people interact with meaning, not computers. People remember Shiva, Ganges and mythology, not abstract bits!

Q: And now we have Eternal Gandhi, a project that our readers can actually go and experience at the Gandhi Smriti Multimedia Museum, New Delhi. Could you tell us more about this project?

A: If you look at the technology aspect, the project demonstrates futuristic tangible learning media. It demonstrates that building technologies that allow people to utilise their innate abilities of physical manipulation and sensing, greatly improves the user interface and learning. With respect to design, the project has carved out a new space of culture-conscious product design and culturally rooted computing, bringing in organic natural material and forms into modern design. It tries to assert the Indian identity, hoping to inspire Indian designers into looking ‘within’ for indigenous forms of expression, and inspire world designers about ecodesign, as well as the need for ornamentation in a homogenised environment. More than anything else, Eternal Gandhi rekindles the Gandhian visions of village creativity, commitment to nonviolence and Gandhiji’s ideas of a multifaceted understanding of truth. The project not only shows Gandhiji as a historical person, but also inspires a new IT vision and modern product design, inspired by Gandhiji as a symbol.
While intellect-based creativity created the atomic bomb, Gandhiji presented us with an approach of non-violence. The project too represents heart-based creativity, the domain of artists, especially village artists. Village creativity is currently under threat, given the homogenising impact of development and global capitalism. The project shows us how village creativity and village materials such as bamboo can be incorporated in the design of computing hardware. As India becomes a first mile IT provider for services and applications within India, and not only a last mile provider for applications outside India, the integration of village creativity and crafts in creating new hardware and software paradigms will become important. The Gandhi exhibition is an imaginative start towards a beautiful vision. It shows how you can weave computing into culture. Imagine having interesting-looking computers instead of staid-looking ones. In the Eternal Gandhi Multimedia Museum all our computers are housed in bamboo or water-proof lacquer cases. Spinning a charkha, touching a pillar or a potter’s wheel and even simply walking alongside a wall act as user interfaces and trigger multimedia presentations at the museum. All these environmental objects, built with the participation of village creativity, act as the UI. If lacquer and bamboo cases can be used to house computers, it means village artisans can also survive out of the IT industry. Therefore, at some point, IT cities such as Bangalore will end up helping not just the US, but also Bangalore’s immediate neighbouring villages. This was the Gandhian vision of the concentric circles of development—let each man help his nearest neighbour! I understand that you are working on designing products (for interaction with computers) that interact with the body to allow people to evoke states of inner intelligence and well-being? This is a dream. Most of us recognise that our bodies are not bags of chemicals but, rather, sites of energy flows and symbolic memory. Most traditional cultures intuit that the mind resides not only in the head, but is also distributed across the body. As a sitar player, I believe my mind is in my hands. The practice of making music, which involves the hands, provides not just music, but health and well being as well. The manipulation of hands thus leads to well being. Hence, if we designed our computers in such a way that someone spending eight hours on a desk actually became spiritually
and physically healthy as a result of interactions with one’s tools of work, “wouldn’t the earth become a paradise,” to quote Gandhiji? For example, I believe (I am not sure because I am not an expert) that interfacing with natural substances like wood, natural fabrics, clay (pottery), etc, create richer and healthful interactions than contact with substances like plastic.
So, natural, lovingly hand-crafted objects would be better to make technological products, which people interact with for several hours a day. Similarly, holding your hands in some yogic postures or mudras is supposed to improve physical and spiritual well-being. Now, your UI device, instead of being shaped as today’s mice are, could be shaped in such a way that when you hold it, your hand is positioned in such an uplifting mudra that you are physically and spiritually uplifted even as you interact with your computer for long hours!

Q: Do tell us a little about your latest projects: Magic Strings of Saraswati, and Rediscovery of the Goddess.

A: Magic Strings speaks to people at multiple levels. At one level it explores the transformation of the Indian lute, the Veena, into a variety of stringed and other musical instruments from India, Myanmar, Cambodia, Thailand, Korea, Java and Bali. At another level, it examines forgotten ideas in analogue-based musical creativity, i.e., the relationship of music to well-being, community, nature and creativity. Yet, at a higher level, it is reasserting Asian dignity, and asserting analogue practices of music in relation to the urrent ‘remix’ musical genre. The Goddess project examines the ‘greening’ of the desktop, and the greening of design. It explores fundamental symbols of the Woman found in all civilisations and translates them into modern design paradigms. This is basically in an abstract form. Womanhood represents the part of the brain that shows a predominance of feelings like compassion, motherhood, intuition, remembrance, etc. In a way, each of our projects is a representation of these.

Q: What do you believe is the future of computer-user interfaces?

A: All the work we’ve mentioned, whether The Crossing, Eternal Gandhi or Magic Strings, illustrate advancements in culturally-reflective computing and product design, and in particular, new interfaces. So, future interfaces will allow us to become healthy, will help us to remember higher ideals, will reflect our identity, will engage us with our bodies and physicality, and will ultimately remind us of larger goals—nonviolence, the recognition of diversity, the remembrance of truth, of our temporary position in time, and the ability to make a meaningful contribution to the earth in the limited amount of time given to us.

© Copyright, Sacred World Foundation, Sacred World Research Laboratory, 2002 - present, All rights reserved.