Fluid Earth: Open-source Visualization of Weather and Climate Data

Introduction

To comprehensively understand weather and climate science, audiences need approachable tools for visualizing atmospheric, oceanic, and terrestrial data. However, the current landscape of online visualizations that can display these sorts of datasets is lacking an option tailored to showing weather and climate systems 1. at the global scale and 2. across the time dimension, all in a simple and engaging manner.

The Fluid Earth tool was designed by the Byrd Polar and Climate Research Center (BPCRC) to present as low a ‘barrier to entry’ as possible for users who want to explore and communicate weather and climate patterns as they change over time. It emphasizes a smooth, intuitive user-interface (UI) for end-users, as well as a streamlined, modularized code-base to make it easier to integrate new datasets and produce custom applications for science communicators.

Null Earth and the Origins of Fluid Earth

The Fluid Earth project originated in 2013 with Cameron Beccario’s Null Earth (https://github.com/cambecc/earth). Null Earth featured an interactive globe that displayed vector-based wind and water stream-lines as well as raster-based color overlays representing a wide range of atmospheric data.

The BPCRC’s Fluid Earth Viewer (FEV, https://fever.byrd.osu.edu/) was developed from Null Earth to make it more accessible for 1. the general public and 2. informal educational settings such as afterschool programs, science centers, museums, and so on. To that end, FEV refined the user interface and added new atmosphere, ocean, cryosphere and climate datasets.

By 2020, however, FEV was showing its age both in terms of user experience and the state of its code. Its rendering engine did not take advantage of modern GPU acceleration, so every user interaction caused a noticeable lag before the display would update.

FEV was also not optimized for slow internet connections, which became an issue for rural users, as well as classrooms where many students all accessed the application at the same time on a single WiFi system. Finally, the application code had grown quite unwieldy, while also lacking modern standards for documentation, version control, and dependencies. All of this made maintenance and expansion of the product increasingly difficult.

Updating Fluid Earth Viewer

In consideration of these factors, we embarked on a project to completely overhaul FEV, accomplished with a team of Research Experience for Undergraduate (REU) students who completely rebuilt the application’s architecture from the ground up.

The new version allows smooth interactions through both space (dragging and zooming) and time (moving across date ranges). It is also optimized for quick downloads, and the code is built according to modern web-development best practices for ease of maintenance and extension.

We originally called the resulting product Fluid Earth Viewer 2.0, but we have since shortened the name to Fluid Earth (FE) to be concise.

Using Fluid Earth

The BPCRC’s instance of FE can be viewed at https://fluid-earth.byrd.osu.edu/. An introductory video tutorial is provided in the Help & About menu.

Fluid Earth for Education

The site can be used as a pedagogical aid, displaying the educator’s preferred date, dataset, projection, and location markers. These settings can then be saved and shared by copying the URL (other users who visit this URL will see the site configured as the educator set it up).

• Fluid Earth displaying dust plumes and convection cells in northern Africa
• Fluid Earth displaying 2021 U.S. wildfires and nearby cities
• Fluid Earth displaying Hurricane Ida’s 2021 Florida landfall

Fluid Earth for Developers

Developers interested in making use of the FE code for their own purposes can fork the Git repository (https://github.com/byrd-polar/fluid-earth). We have also included a lightweight web component for use in webpage contexts where the full application is not needed; introductory documentation and a demo page are included in the repository to help developers get started with the web component.

Materials and methods

Our team made careful consideration of code architecture before starting our modernization of Fluid Earth. Our approach combined a complete overhaul of the map-rendering engine (in WebGL) with a modern user-interface framework (Svelte) and a completely rebuilt back-end that was organized for easy management and addition of data layers.

The Fluid Earth application is built around a world-scale map of Earth which is overlayed with color-coded weather bitmaps and animated wind/ocean-current streamlines.

The original implementation of the map was throttled by the speed of users’ CPUs. After any user interaction such as zooming or panning, the image would take several seconds to refresh, which introduced large amounts of friction to the user experience.

Rebuilding Fluid Earth

We rebuilt the engine in WebGL (https://www.khronos.org/webgl/) to dramatically accelerate rendering using the GPU. Fluid Earth can now maintain seamless projection of raster data as the map is dragged and resized, while also animating hundreds of thousands of particles without slowdown.

For the user interface surrounding the map, we opted to use Svelte (https://svelte.dev/), a UI framework that is focused on modularity. This allowed us to develop a very well-organized application, with each on-screen component (menus, buttons, map keys, etc.) represented by its own group of files with only a few dozen lines of code each (instead of hundreds or thousands in the previous version).

Svelte also automated low-level event processing, freeing us from the fine-grained details of implementing particular controls so we could focus instead on design questions. This reduced complexity, made bug-shooting significantly easier, and greatly expanded our ability to build an interface that users would find intuitive and pleasant to use (and refine it in response to iterative feedback).

We also rebuilt the data back-end of Fluid Earth, organizing it around an inventory containing all of the metadata for each data layer. This allowed us to simplify the format of the data files themselves, reducing download times (especially for bandwidth-constrained users) by reducing file size. This architecture also makes it considerably easier to add a new custom data layer; users simply generate the data in .fp16 format (just an array of 16-bit decimal numbers) and add a new entry in the inventory file for how to display it.

Finally, we also modernized our approach to development in general, making use of a suite of web development tools: Git (https://git-scm.com/) for collaborative revision control, Node.js (https://nodejs.org/) for dependency management, Vite for front-end build (https://vitejs.dev/), and so on.

We strove to avoid the sprawl of ‘spaghetti code’ that plagued our previous development. In sharp contrast to our earlier work – which was carried out by individual programmer-specialists each inheriting the code, and the problems, of their predecessor – our practices when developing the new Fluid Earth focused on intercommunication between the team of REU students and center staff, which  benefited from the integration of diverse talents and experience levels to produce a high-quality product while enriching each team member’s skillset in return.

Throughout development, the team used information gathered during prototyping with youth and families, outreach events with the general public, and feedback solicitations to educators to add and refine features. Each of the new features and improvements was made in response to a specific issue identified by the team during user-testing or specifically requested by educators.

Outcomes

The new Fluid Earth improves markedly on its predecessor in several ways.

Weather variables are displayed at approximately 30 km resolution over the whole Earth’s surface. (See Tables 1a-1c for a full list of variables.) On top of the color data are streamlines moving at proportional speeds to wind or ocean currents.

Wind data is available at several elevations throughout the atmosphere and is automatically altitude-matched to the raster data when switching between weather variables. Users can quickly switch between weather variables, select specific dates, or step back and forth through time hour by hour.

The globe itself can be viewed in a variety of different projections, and specific points can be investigated using location markers and a city search tool.

Table 1a: Weather Data Category

Data layer	Type	Unit*	Elevation(s)	Temporal resolution	Source**
Temperature	Raster	°C	500mb, 200mb, 10 m a	Hourly	GFS
Wind speed	Raster	km/h	500mb, 200mb, 10 m a	Hourly	GFS
Wind streamlines	Vector	km/h	500mb, 200mb, 10 m a	Hourly	GFS
Humidity	Raster	%	500mb, 200mb, 10 m a	Hourly	GFS
Precipitation	Raster	kg/m²	Surface	Hourly	GFS
Pressure	Raster	hPa	Mean sea level	Hourly	GFS
Water in atmosphere	Raster	kg/m²	Entire atmosphere	Hourly	GFS
Water in clouds	Raster	kg/m²	Entire atmosphere	Hourly	GFS
Sunshine	Raster	min/h	Surface	Hourly	GFS

Table 1b: Gases and Aerosols Data Category

Data layer	Type	Unit*	Elevation(s)	Temporal resolution	Source**
Ozone	Raster	DU	Entire atmosphere	Hourly	GEOS-5
Sulfur dioxide	Raster	μg/m³	Surface	Hourly	GEOS-5
Carbon monoxide	Raster	ppbv	Surface	Hourly	GEOS-5
Dust	Raster	μg/m³	Surface	Hourly	GEOS-5

Table 1c: Ocean Data Category

Data layer	Type	Unit*	Elevation(s)	Temporal resolution	Source**
Sea temperature	Raster	°C	Surface	1 day	RTGSST
Currents	Raster	km/h	Surface	5 day	OSCAR
Wave height	Raster	m	Surface	Hourly	OSCAR
Ocean currents	Vector	km/h	Surface	Hourly	OSCAR

* Native units for each data source are shown, but where applicable, conversions to Imperial and other systems of measurement are enabled in Fluid Earth.

** GFS: Global Forecast System. GEOS-5: Goddard Earth Observing System. RTGSST: National Centers for Environmental Prediction Real-Time Global Sea Surface Temperature. OSCAR: Earth Space Research Ocean Surface Current Analyses Real-time.

Sharing Event Specific Map Views

Importantly, the configuration of the entire application is continually updated in the address bar. This allows educators to easily highlight particular phenomena and events by sharing a URL.

For example, a link could display a view of sulfur dioxide plumes in the American west on a date of particularly intense wildfires, as well as location markers highlighting the sulfur dioxide levels in several cities in the region, allowing audiences to easily see the effect of the plumes on population centers.

Optimized Map Views for Both Mobile and Desktop Browsers

During user-testing and correspondence with educators, we became aware that learners benefit substantially when allowed to interact with and explore the data freely on their own (Cervenec, et al. 2021), and that the less friction was present in those interactions, the more engaged those learners will be.

Accordingly, Fluid Earth has been optimized for smooth interactive performance on both desktop and mobile browsers. User testing in classrooms also alerted the team to the issues that arise when 30+ students each try to use a bandwidth-greedy program on a school WiFi network simultaneously.

Fluid Earth features reduced file sizes and speedier downloads, which also benefits individual users in rural and remote locations that may lack fast broadband offerings. (See Table 2 for performance comparison between FEV and FE.)

Table 2: Benchmarks for FEV and FE

Benchmark	FEV	FE
Initial page size*	3.0 MB	4.7 MB
Page load time (Time to Interactive)**	3.2s	1.7s
Number of wind/current particles	4,608 (all visible)	100,000 (not all visible)
Raster data upate time after rotating globe*	~900ms	50ms (steady 20fps)
Performance score**	78	75

Introducing a Time-lapse Function

Another addition of particular value is the new Time-Lapse function. By clicking a button, users can pre-load data in bulk for an extended period of time (ranging from hours to years), then play it back as a smooth, looping animation.

This permits exploration of dynamic phenomena that are not readily apparent from still images or by laboriously selecting and downloading timepoints one-by-one; for example, the daily oscillations of temperature at a given latitude, convection cells rising up over savannahs, dust drifting in plumes across the Atlantic, and tropical storms progressing toward landfall.

Finally, we also prioritized customizability and extensibility in the new Fluid Earth. The code is easy to re-implement in any Windows or Linux environment. It comes with data downloader functions for the full set of data layers in use by the BPCRC, and users can easily add to this base list using the back-end inventory.

The Svelte framework allows for modular addition of new features to the front-end for specific applications.

Fluid Earth also comes with a lightweight, easy-to-customize web component version which can be embedded in webpage contexts where the full application is not needed. These tools have already been deployed with the team’s Virtual Ice Explorer tool (https://virtualice.byrd.osu.edu/) and by collaborators as part of the Permafrost Discovery Gateway (https://arcticdata.io/catalog/portals/permafrost/).

Discussion

It is worth considering what unique features Fluid Earth brings to the ecosystem of similar online weather- and climate-visualizing map tools such as Windy, Ventusky, and Meteoblue (see Table 3 for comparisons).

Table 3: Comparison of Weather and Climate Visualizing Map Tools

Product	URL	Rendering	Weather detail	Projection	Animation FPS
Windy.com	https://www.windy.com/	2D tiles	Tens of miles	Rectangular	1
Ventusky	https://www.ventusky.com/	2D tiles	Miles	Rectangular (globe available in mobile app)	1
Meteoblue	https://www.meteoblue.com/en/weather/maps/	2D tiles	Miles	Rectangular	N/A
AllisonHouseMaps	https://maps.allisonhouse.com/free/maps	3D tiles	Tens of miles	Rectangular, globe	10+
Google Earth	https://earth.google.com/web/	3D tiles	Hundreds of miles	Globe	1
Fluid Earth	https://fluid-earth.byrd.osu.edu/	Whole earth	Tens of miles	Rectangular, Globe	10+

These other tools use tile-based rendering, meaning they load sections of the map progressively, allowing users to drill down to high-resolution imagery. However, tiled data adds considerable complexity to the application logic and can lead to a sluggish experience for the user as they wait for sections of Earth’s geography to load in.

Many tile-based tools such as the above are also only able to support Mercator-style (rectangular) projections. Products such as Google Earth and the Cesium-powered AllisonHouseMaps (see Table 3) which utilize polygon-based mesh tiles, are able to get around this limitation and even render street-scale landscapes in realistic 3-D. However, these are even more complex than 2-D tiles and may be less performant for users with poor bandwidth or underpowered computers.

Fluid Earth’s Unique Features

With Fluid Earth, we aimed to serve a specific niche: exploring weather and climate phenomena at global scale, focusing on change over time, with a premium put on smooth interaction, accessibility, and ease of use for educators, learners, and developers.

Fluid Earth is able to depict the world as a globe in addition to rectangular or other projections (which also allows polar regions – an area of focus at BPCRC – to be displayed without the extreme distortions common to rectangular projections). The datasets we include (see Table 2 for data sources) were specifically chosen to be transnational and extend across the full globe.

Considering this global scale, we chose a more lightweight, lower-resolution rendering solution that did not rely on tiling. Our application loads data for each variable and timepoint all at once, resulting in a slightly longer initial load time in exchange for dramatically improved after-load performance. Load times are minimized by keeping file sizes small, and the Time-Lapse feature further minimizes the effect of loading data when exploring a defined interval of time. Our smaller file sizes also mean that Fluid Earth performs well on low-bandwidth or low-performance machines and on mobile devices.

Fluid Earth Target Audience

We targeted Fluid Earth at an audience of the nonexpert public, educators, and science communicators, and designed it to be accessible to a general audience including under-served groups and youth as young as elementary school (for results user testing with 6^th an 7^th grade audience, see Cervenec et al., 2021).

Fluid Earth features a simple, easy-to-understand interface with generous real-estate dedicated to exploring change over time, in order to facilitate learning about weather and climate as dynamic systems. Fluid Earth is not intended to provide a platform for deep, technical interrogation of these topics, but to provide educators and science communicators with a simple, high-level visualization for engaging introductory lessons on atmospheric patterns and discreate weather events.

While outside Fluid Earth’s target audience, there has been interest in its use with students of atmospheric sciences and experts in other disciplines who are new to exploring weather/climate data, where it can serve as a stepping-off point for subsequent explorations using more complex and application-specific products.

We also aimed to serve computer-adept educators who may want to deploy a visualization tool for sharing their own lessons and data sets, but don’t necessarily have the resources to create that tool from scratch. Our application is open-source, is easy to deploy and customize, and includes a web component which is easy to embed in a webpage without having to implement the full application.

Lessons for student-based software development paradigms

The original Fluid Earth Viewer was built in an ad-hoc manner. Each developer worked largely in isolation, writing idiosyncratic code to quickly add small features or spot-fix bugs as they occurred. This approach neglected important practices to ensure sustainability, such as periodic documentation, code restructuring, and forecasting future needs.

We built up ‘technical debt’ to the point where remediating the old code was not feasible. Our only realistic alternative was a ground-up rebuild, which would also give us the opportunity to reimplement best practices.

To achieve this in the context of the COVID-19 pandemic, we attempted an experiment: an NSF-supported project to redevelop a web-application in a largely remote work environment, using a team of Research Experiences for Undergraduates (REU) students.

Depending on the time of the semester, our team comprised 3-5 students, each of whom was trained on the job in the basics of web development. The team manager and one of the students (a subject matter specialist) served as coordinators, administering collaborative tools (such as the GitLab infrastructure), alerting the team to best coding practices, and scaffolding skill acquisition and task pools for the group. Within this structure, individual students were allowed a measure of self-direction when selecting project areas to work on.

Throughout the development cycle, we also met regularly with our outreach coordinator, Jason Cervenec, for guidance on tailoring the tool for scholastic educators and our atmospheric scientist, Aaron Wilson, for best practices in the visual representation of weather and climate datasets.

A large amount of attention was directed towards communication in the constraints of remote teamwork, with a Basecamp task-management system, daily online check-ins, and weekly Zoom meetings supplemented by focused calls to troubleshoot blockages or help a developer plan out a more complex task.

One lesson we learned as the program went on was that our initial approach, inspired by corporate-oriented ‘Agile’ and ‘nimble’ collaboration philosophies, needed recalibration for a university environment. We retained the focus on cyclical assessment of the team’s progress combined with regular re-planning based on empirical evidence. However, we recognized the need to relax the system somewhat and empower our personnel (students with multiple obligations and constraints on availability) with more flexibility and autonomy.

As the team gained competence and confidence over the course of the semester, our early meeting structures gave way to more frequent self-directed work, with responsive attention to specific problems.

Given these practices, we found that our REU students were more than capable of producing a high-quality product that markedly improved on our previous version of FE. (It should be noted that it was one of the REU students themselves who proposed the code overhaul in the first place, demonstrating the high level of competence and initiative taken by the group.)

Considering the diverse backgrounds, personalities and skillsets that each team member brought to the project, we believe that these results could generalize to future remote-development projects in the same vein.

Future work and collaborations

Fluid Earth is an evolving project and there are a number of avenues for future development, including: adding additional data sets to serve educators teaching atmospheric sciences and oceanography, including long-term climate averages and anomalies; potentially adding higher-resolution datasets, including those focused on polar regions; the ability to display multiple raster data layers simultaneously; and improved accessibility for visually-impaired audiences.

New features may be initially tested in ‘Advanced mode’, which is accessible via a toggle switch on the main menu and which gives access to both experimental features and features targeted to a more expert audience that is emerging in our work with the Permafrost Discovery Gateway (https://arcticdata.io/catalog/portals/permafrost/).

We also strive to keep Fluid Earth continuously relevant by expanding its applications to new educational contexts and to better serve both broader impacts and research communities.

If you are interested in making use of Fluid Earth, either by presenting the main application (https://fluid-earth.byrd.osu.edu) or by adapting the full site code or web component (https://github.com/byrd-polar/fluid-earth), please reach out to the BPCRC project team for support and to discuss possibilities for collaboration.

Lastly, the project team was inspired to offer virtual summer REU opportunities during the COVID-19 pandemic. While the project team was initially hesitant to embark on this venture, we had employed undergraduate students in the past, had a track record of offering students independent areas of investigation within our funded projects, and were highly motivated to delivery rewarding experience during a time of extreme disruption to student lives.

With the support of our NSF program manager, Lisa Rom, we took the plunge and offered something new. While we had a solid plan that included synchronous and asynchronous communications, regular discussion of personal and institutional goals, rich opportunities for skill development, weekly mentor meetings, and a Team Science workshop, we also had to make adjustments on the fly.

The experience was rated as positive by both students and the project team, inspiring us to plan its expansion in a third summer. In addition to offering students who have little programming experience the opportunity to learn, it provides opportunities to develop skills in prototyping and user testing, project management and version control, internal and external communication, and iterative design between content experts, coding professionals, and end-users.

While the virtual REU experience was created for the pandemic, it also benefits non-traditional students who are not able to relocate for the summer and allows limited funds to support a greater number of students (while students were paid an hourly wage for the summer experience that totaled $6,000, there was no need for travel costs, food, or lodging).

We acknowledge that there are still benefits of in-person collaboration, and hope that when the pandemic recedes we could explore the best blend of virtual and in-person experience that still afford for some of the increased access we have been able to provide.

For externally-funded, academic research projects, the REU students offered other benefits, such as an ability to have a diverse team of individuals work on a focused series of deliverables that are limited in duration. Undergraduates often arrive with approaches to solving problems and skills that afford teams a fresh perspective, as has been our perspective. 

Acknowledgments

The authors would like to acknowledge the contributions of Julien Nicolas in creating the first database and user interface for FEV, ongoing work by Bidhyananda Yadav and Aaron Wilson to maintain the FEV/FE datasets, Tom Kassebaum and Pam Theodotou for technical expertise, and the talented group of undergraduates and REU students who have assisted in designing the user interface, conducting user-testing, organizing data, and producing tutorials: Bingyu (Sophia) Li, Dingyu Hu, Ruiyang Chang, Joey Wong, Craig Bossley, Mimi Cai, Adelyn Arens, Devan Agrawal, Daniel Hamilton, and Shaniqwa Martin.

The open source code that was the starting point for FEV was created by Cameron Beccario whose Earth visualization is available online at https://earth.nullschool.net/.

Declaration of interest statement 

The authors declare no competing financial interest or benefit that has arisen from the applications of this work.

Funding information 

This work was supported by the National Science Foundation under Grant number 1612741.

ORCID

Michael Gravina http://orcid.org/0000-0002-8385-5113

Jason Cervenec http://orcid.org/0000-0003-2608-1198

Data availability statement

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

References 

Cervenec, J., Fox, J., Peggau, K., Wilson, A.B., Li, B., Hu, D., Chang, R., Wong, J., & Bossley, C. 2021. “Interactive Data Visualizations of Earth’s Atmosphere: Effects on Student Engagement and Perceived Learning.” Manuscript submitted for publication.  

About the Authors

• Michael Gravina is a Systems Developer/Engineer at the Ohio State University Byrd Polar and Climate Research Center, leading web development efforts for Education and Outreach and coordinating the Center’s team of Research Experience for Undergraduates students. Michael earned a B.S. in Environmental Science from Clark University and an M.A. in Biology from the Harvard University Extension School. He has worked in a variety of fields including pharmaceutical manufacturing, genomics, computational psychology, medical research, sports analytics, and climate outreach, striving to make information more accessible and complex systems easier to understand and use across each of these environments.
  https://orcid.org/0000-0002-8385-5113
  https://www.linkedin.com/in/michael-t-gravina-03b62a21/
  *Corresponding author. Email: gravina.2@osu.edu
  
• Michael Zhan is a front-end and full-stack developer who served as an NSF Research Experience for Undergraduates intern with the Byrd Polar and Climate Research Center’s Education & Outreach Team from 2020 through 2021. During this time, Michael distinguished himself as a leader on the Fluid Earth application rebuild project, outlining the development plan, serving as subject matter expert at weekly strategy meetings, educating other team members on relevant technologies and methods, designing back-end and UI architecture, and authoring a majority of the new code for Fluid Earth.
  https://www.linkedin.com/in/michaelemilzhan/
  
• Mimi Cai is a 4th year student at the Ohio State University majoring in Data Analytics and minoring in German. She discovered her passion in quantitative analytics while taking a data visualization class through the political science department and has continued to take an interdisciplinary approach to data, pursuing undergraduate research in different academic fields. Her eclectic interests range from data in public policymaking to natural language processing. She currently plans on continuing her studies through graduate school in Computational Linguistics.
  
• Adelyn Arens is a third-year Data Analytics and Economics major with a minor in Leadership Studies at Ohio State University. She has been working at the Byrd Polar Climate and Research Center since summer 2020 on both the Fluid Earth Viewer and the Virtual Ice Cap Tours. In addition to Byrd, Adelyn has been involved in a variety of analytics and research experiences such as interning for the Cleveland Clinic as a business analyst (Summer 2021), authoring a thesis paper on youth sports and parent-child relationships, actively participating in the Big Data Analytics Association, and interning as a business analytics intern at Cisco (Upcoming- Summer 2022). Her future plans include potentially continuing on in analytics at Ohio State after graduation in either the Industrial Systems Engineering or Applied Statistics Masters programs.
  www.linkedin.com/in/adelyn-arens-026158192
  
• Jason Cervenec is the Education and Outreach Director for the Byrd Polar and Climate Research Center at The Ohio State University, whose outreach portfolio reaches approximately 12,000 individuals annually and includes programs in cutting edge science, science education, history, and the arts. Jason earned a B.S. in Biology and M.Ed. in Secondary Science Education from The Ohio State University. He taught for more than a decade as a high school science teacher, established two Science Olympiad teams, served as a lead instructor on an Ohio Board of Regents grant to train teachers in Modeling Instruction, and took part in a Fulbright Teacher Exchange in Mumbai, India where he witnessed firsthand the challenges faced by most of the world’s population. He currently leads the Columbus Climate Change Action Plan Task Force and volunteers with Franklin County’s Restorative Justice Circles.
  https://byrd.osu.edu/people/cervenec.1

Related

• Earth Wind Map
• Wind and GIS
• Geospatial Climate Models Show How Fire Seasons Are Changing