By T.H. Culhane, Ph.D.
Director, Climate Change Mitigation and AdaptationUSF Patel College of Global Sustainability
Co-founder and director, Solar CITIES Inc.
November 25 2018
Photo: An early virtual reality interactive game version of Rosebud Sustainability Education Center in development showing Puxin biodigesters placed in the landscape with a playable avatar.
Photo: An early view of the actual Rosebud Sustianability Education Center biodigestors and aquaponics under construction.
CLOSING THE METABOLIC RIFT THROUGH VIRTUAL REALITY
“Oculus Rift”, “Oculus Go”, “Sony PlayStation VR”, “HTC Vive”,“HTC Vive Pro”, “Samsung Gear VR”, “Lenovo Mirage Solo With Daydream”, “Google Cardboard” and now “Magic Leap” -- these strange sounding names are becoming household words as consumers begin to embrace virtual and augmented reality as THE future of entertainment . Now producers of content for these platforms are moving way beyond entertainment and creating and envisioning applications for immersive 3d experience in everything from military training to medical procedures.
What started as a niche area for “gamers” has now become the hottest human-machine interface, destined to replace the computer screen, TV monitor, keyboard, clicker and mouse in virtually ALL areas where we engage with digital and audio-visual content.
One of the questions that educators in general are asking as we enter the brave new world of virtual and augmented reality is “when young people put on their VR headsets, what will they be seeing? What will they be experiencing? How will the experience in virtual reality shape their perceptions of and behavior in ACTUAL reality?”
The more pressing question for us as educators in sustainability is “can we use these technologies to help close the metabolic rift - the divide between natural and anthropogenic industrial systems -- so that instead of using VR technology to escape from reality, students can use it to improve reality”.?
We want to know “given the state of the world today, and the urgent need to preserve biodiversity and endangered landscapes and seascapes, maintain food security and combat poverty in the face of population growth and global warming, are we using today’s technology in the right way and are we doing enough to prepare students and the next generation for this transition so that the power of virtual reality can be harnessed and applied in time to meaningfully mitigate and adapt to climate changes and environmental degradation?”
Shouldn’t we be LEADING THE CHARGE to create virtual reality experiences that can shape more sustainable practices in the real world?
The Patel College/Solar CITIES ADVENT initiative:
“Advanced Data Visualization for Education in Nexus Thinking”
With our USF Patel College of Global Sustainability “Advanced Data Visualization for Education in Nexus Thinking” Initiative (The “ADVENT” Initiative) led by Dr. T.H. Culhane, director of the Climate Change Mitigation and Adaptation Concentration and co-founder of the not-for-profit organization “Solar CITIES”, which concentrates on Food/Energy/Water Nexus and Zero Waste education around the world, we are seeking to create a new concentration called “ENVISIONING SUSTAINABILITY” where one of the chief goals of the program is for sustainability students to learn how to MODEL ALTERNATIVE FUTURES and COMMUNICATE best practices and create action and hope inspiring narratives.
A major part of the “Envisioning Sustainability” Program involves student mastery of today’s communication technologies for video and multi-media production, website and blog creation, digital photography, and animation to create compelling content for public service announcements, public presentations, poster sessions, scientific papers and multi-media content focused on sustainability. But our ADVENT initiative goes even further, embracing the trends to incorporated VR, augmented reality and the “internet of things”.
Students in the program will learn how to model the world around them using the latest 3d scanning and 3d printer hardware and the most powerful data collection, analysis and visualization software, and will learn how to gather and process environmental data to build interactive 3 dimensional models of critical locations so that various scenarios of sustainability (or lack thereof) can be simulated, demonstrated and communicated -- and even “played” as a kind of interactive viscerally intuitive and impactful “game”.
The premise of the program is that we can immediately harness the immersive data visualization capabilities of today’s 3d enabled Advanced Geographical Information Systems (GIS) and blend it with 3d scanning hardware, and mesh-modeling, and animation and physics engine/gaming software and create “playable” avatars of our actual student students and faculty and interactive digital models of the actual homes, neighborhoods, communities, institutions, and landscapes surrounding our actual students and faculty that can be visited in VR and walked and flown through and, most importantly, TRANSFORMED BY THE STUDENTS AND FACULTY AND LEARNING COMMUNITY to enable all to ENVISION alternative scenarios resulting from the implementation of various technologies and policies so that we can better decide what is appropriate to implement in the real world.
LEARNING REAL WORLD SUSTAINABLE DEVELOPMENT TECHNOLOGY, VIRTUALLY
“In game” students will be able to get virtual “hands-on” training in the construction of system integrated community and home biodigesters, solar hot water and photovoltaic systems, hydroponic and aquaponic and aeroponic food “soil-free” food production systems, geothermal heat pump systems for heating, ventilation, and cooling, small wind and micro hydro systems, weather, hurricane and earthquake resistant buildings and other disaster preparedness, climate resiliency and permacultural infrastructure. They will learn systems thinking and nexus thinking and have a chance to engage in no risk trial and error experiments to decide what configurations and applications work best in various environments of relevance and interest to THEM. This will prepare them for work in the real world such as Solar CITIES has been conducting in developing regions, but without any risk or danger, and will give them a chance to see how “best practices” can be extended to their own homes and communities.
LEARNING REAL WORLD DATA COLLECTION AND ANALYSIS, VIRTUALLY
In addition to learning how to implement technologies for sustainable development in the virtual and augmented reality of gamified environment we will create “in-game” tools for data acquisition that will help train students how to construct, apply and use data collection tools in the real world. Students will be able to build environmental sensors and implement them virtually that will help them understand how to construct real life analogues of the same thing. For example, in VR students will put together and program Arduino and Rasberry Pi sensing kits so that once they have completed a game level they know how to do the same thing in real life without worrying about damaging something or doing it wrong. This is the promise of VR and augmented reality for training.
ATTENTION TO PERSONAL NEEDS, INTERESTS AND NARRATIVES
One of the things that makes our initiative at USF stand out is our commitment to “multiple personalized narratives” and personal playable story arcs CUSTOMIZED to each student’s experience through “relational student empowerment developed sustainability storytelling” . We use the technology to augment the lived reality and real spaces of our students so that they can transform THEIR environments into multiple “what if” scenarios and simulations that can communicate alternative futures relevant to them and their communities. In other words, when students come to our college they will scan themselves into the virtual world as playable avatars and then scan in their own home and school environments and other landscapes important to THEM and populate those landscapes with the mitigations and adaptations they want to research for making these environments better or with technologies and infrastructure that they worry could make things worse.
The program and initiative will enable the Patel College to attract, recruit and retain an ever wider student body, particularly the millions of active “gamers” who are already excited about playing various fictional “save the world” and “survival” campaigns (one of the most popular games on the internet currently is the “Fortnite: Save the World” campaign) and have longed for a program that blends the virtual and the real toward the goal of truly “saving the world”.
Working with the leadership of Dr. Laura K. Harrison, director of the USF Access 3D Lab and researcher in the Center for Virtualization and Applied Spatial Technologies (https://www.usf.edu/arts-sciences/research-scholarship/documents/access…), myself and Dr. Brooke Hansen, director of the Sustainable Tourism program at Patel College are already training students to digitize objects and landscapes. Dr. Hansen and her classes are using drone based aerial photogrammetry and mobile scanning equipment to virtualize a historically important and ecologically sensitive Seminole Indian burial ground and former concentration camp at Egmont Key National Wildlife Refuge off the coast of Tampa that is being impacted by climate change and coastal erosion while I and my students are digitizing the Rosebud Continuum Sustainability Education Center in Land O Lakes, where students have already built several real life 9 foot tall food-waste biodigesters and aquaponics and hydroponics systems that are powered by solar and wind energy on a landscape being restored to native vegetation.
SIMULATING LOCAL IN-SITU SUSTAINABILITY
Our goal is to continue virtualizing important local landscapes and features in the field work radius (within an hour’s drive) of the Patel College, and in partner locations around the world identified by our students and faculty through their internships and Academic Capstone Experiences, and USF World and global university and NGO partnerships so as to create an interactive virtual simulacrum model of all the spaces relevant to Patel student projects in sustainability.
Once the spaces are digitized and “put on the map” (figuratively and literally) alternative futures for these areas that take into account the climate and environmental effects of the anthropocene can be modeled and different planning scenarios simulated.
On-line students as well as in class and in-field students can meet in these digital simulations and work together in groups to apply targeted UN Sustainable Development Goals at the local level in environments with which they are familiar and thus solve real world and often difficult and dangerous problems in safe virtual spaces. Students will learn, for example, how to build virtual biodigesters and install virtual renewable energy systems and plant and maintain vertical soil free food production systems, engaging in real time systems thinking and nexus thinking that can be “fast forwarded” to experience the outcomes of various system configurations before committing to real world implementations. Virtual reality is emerging as a powerful planning tool in architecture, city planning and engineering, and our students will be able to use the same scenario building tools in the service of sustainability planning.
Using the now popular VR chat features already built into the new VR platforms, our students will be able to do powerful group work and meet with each other and with students around the world in real time, and be able to engage in meaningful field training experiences together to prepare them for actual site visits later.
Ultimately, we plan to merge all of the files and experiences into a “massively multiplayer online role-playing game (MMORPG)” that we call “Masters of Sustainability” where students can “play” their way toward mastery of the core principles of sustainability as they in fact earn their “Masters Degree in Sustainability”. We have already scanned faculty and students into virtual interactive playable avatars using Dr. Harrison’s Photogrammetry and Culhane’s work with the open source creativity software platforms Blender 3D and Unity 3D, and created interactive landscapes for the Rosebud Center using QGIS files and 3D sustainable development objects from Google 3D Warehouse.
Students in our program are already being trained to “composite” or “mash up” these data sets in Dr. Harrison’s computer lab and in the Digital Media Commons at the USF library where our university supplies a green screen/virtual reality room and creative workstation center and where trainings are offered in the necessary software for video and VR content production. Dr. Harrison is offering our students career enhancing “badges” for 3d virtualization training completion and certificates in GIS and other relevant software applied to the sustainability field are offered through Patel and USF.
We are also collaborating with former Patel Climate Director Dr. Seneshaw Tsegaye, now running the VIPER VR lab at Florida Gulf Coast University (see https://www.fgcu.edu/eng/softwareengineering/viperlab/index.aspx) , whose students are already using VR and programming in Unreal Game Engine and working with augmented reality to model their campus and simulate climate threats. These collaborations give us a wide network of dedicated developers and access to the expertise necessary to sustain this effort.
With a program in “Envisioning Sustainability” students would be prepared for traditional careers with environmentally focused media corporations like National Geographic, Cengage Learning, the Discovery Channel, Disney Nature, Scientific American and the Smithsonian, to name just a few of the giants in science communications. There are also numerous educational publishers and media groups and non-profits involved in creating science and educational and environmental advocacy content, to say nothing of traditional news and media outlets which rely on scientifically literate and well trained communicators and content producers. This is to say nothing of the huge demand in the billion dollar film and video game industries, which seek to create believable alternative future scenarios, some eutopian, some dystopian, but all based on extrapolations from the real world, for its speculative fiction offerings. And this is without including the robust job market for urban and rural planners, green architects and environmental project managers.
The bottle line is that jobs abound for sustainability students who have the skill sets to “envision sustainability” and demonstrate unsustainable scenarios in compelling ways. However to date there are very few graduate programs that prepare students for such careers.
IMMERSIVE SUSTAINABILITY DATA VISUALIZATION IN YOUR OWN HOME TOWN
What makes the ADVENT Initiative relevant and impactful is our focus on creating augmented reality and virtual reality content based on the real needs and landscapes of our student body and faculty and communities. Because we are an international “college of GLOBAL Sustainability” we seek to create a patchwork of virtual environments based on real locations that students “in-game” can teleport to for doing remote virtual research and for engaging in “compare and contrast” learning and sharing.
Six sites have been selected to begin the initiative based on existing agreements and interests.
Rosebud Continuum Sustainability Education Center, 22843 Hale Road, Land O Lakes FL 34639 (Pasco County)
This is a family owned and community supported sustainability center featuring permaculture technologies and rewilding where school kids come to learn about how humans and nature can coexist and where Professor Culhane lives off grid using solar energy and biogas.
Fat Beet Farm, 13802 W Hillsborough Ave, Tampa, FL 33635
This is a family owned property where Patel Students intern to grow organic food that is processed for 5 restaurants owned by the Curci family who run the Noble Crust restaurant chain (http://noble-crust.com/) and where best practices in zero waste are being implemented.
Egmont Key National Wildlife Preserve and Seminole Alliance at Egmont Key Island, The Egmont Key Alliance P. O. Box 66238St. Pete Beach, FL 33736
This is a historical monument where the Seminole people of Florida made a stand for their freedom from colonialism, were conquered and incarcerated. It is being impacted by tourism, development and climate changes which USF students are studying.
The Dr. Sylvia Earle historic property at Earle Lake in Dunedin, Dunedin, FL 34698
This is the childhood home, forest and lake property of world famous oceanographer and National Georaphic Explorer in Residence Dr. Sylvia Earle, which her family has fought to preserve in its native Florida “wild” state against urban encroachment for generations, but is being fought by developers who want to make condominiums. USF students are volunteering there to identify wildlife and create an education center.
Bonnekamphohe Stiftung, an urban “Transition Town” Permaculture farm, Bonnekamphöhe 17 45327 Essen Germany
Run by visionary environmental educator Hubertus Ahlers, this is one of the last remaining tracts of land where best practice models of integration between human food and energy production and wilderness are being taught through cooperation with Dr. Andre Matena and his students from the University of Essen, Germany, and the Patel College with community members and students and faculty of both Universities given opportunities to volunteer and intern and learn about Urban Resiliency, Food Security and the Food Energy Water Nexus.
Suderbyn Ecovillage, Västerhejde Toftavägen 211
Visby, Gotland, Sweden
Suderbyn is a living sustainability community with over 20 residents where “life tested” experiments in ecological solutions are tested and integrated into daily life. Students from around the world are given a chance to do research in this “playground” of ideas.
The number of digitized landscapes will grow with each semester and with the involvement of more institutions and partner faculty from around the world, such as Dr. Seneshaw Tsegaye from FGCU and Dr. Andre Matena from the University of Essen in Germany.
Aerial drone photogrammetry will be conducted first to get the lay of the land and then finer details will be scanned in using terrestrial laser scanners.
The data clouds will be processed, meshed and textured and rendered into usable virtual environments that will be geocoded as modules on an evolving world map of interactive “hotspot models for sustainability education”.
Students will then take the textured mesh data and add data layers of existing infrastructure and landscape features and be able to place object meshes of best practice sustainability technologies and practices for running scenarios. Students will also scan each other into Avatar meshes that they can put into the scenarios and control as if they are in a game. With VR students will be able to inhabit the avatars (as experienced in the popular immersive VR platform “The Void”) and interact with each other for team building and collaborative research and role playing.
Variables for environmental forces such as storm winds, cold snaps and heatwaves, and floodwaters and drought induced fires, and vegetation die-offs, as well as infrastructure challenges such as power outages, downed transmission lines and damaged pipes, roads and buildings will be added so that students can “play” various future outcomes from their choices.
In each “hotspot” environment added to our world map, students will have a chance to use virtual and augmented reality to teleport from site to site and visit best practice ideas being implemented right now, virtually meet the people who are creating them on the ground and envision together what could be there, sharing information in real time with other student “players” around the world and playing the different outcomes for better planning. .
The goal of the final product is to develop an immersive interactive massive multiplayer online role playing game called “Masters of Sustainability” that will act as a valuable planning tool and allow students to play their way toward mastery of sustainability concepts, learn real world technological skills and earn their “masters of sustainability” degrees from USF.
BADGES AND CAREER READINESS
Students in the Envisioning Sustainability ADVENT Initiative will earn technology badges from the USF Career Services Badging Program (https://www.usf.edu/career-services/career-ready/badge-career-managemen…) and will be industry ready and prepared for careers in fields that involve:
• Climate Change Mitigation and Adaptation
• Sustainable tourism applications
• Urban Planning for Sustainability
• Wildlife Conservation
• Coastal Management
• Virtual museum collections
• Architectural and landscape modeling
• Mapping and georeferencing
• Reverse engineering
• Design and prototyping
• Die and mould design or modification
• Forensic analysis
• Digital reconstruction and conservation
• Cultural heritage preservation
• Quality control
• Prosthetics, orthopedics, and plastic surgery
Example Software used and taught in the initiative:
Artec Studio 13: Industry acclaimed software for professional 3D scanning and data processing
Adobe Creative Suite
Hardware used and taught in the initiative
Phantom 4 DJI drone: aerial drone equipped with high resolution camera and 4K video.
FARO Arm: laser scanner for small to medium sized objects, precise measurement without color.
Photogrammetry studio: Canon 5DS 50 megapixel cameras with peripherals; creates highly realistic 3D models; small to large objects and buildings.
FARO x330 & s150 (not pictured): terrestrial laser scanners for architecture and landscapes; full color capture.
Artec Eva: structured light scanner for larger objects; full color capture.
Artec Space Spider: structured light scanner for smaller objects; full color capture.
Arduino Environmental Sensing Kit