When a prominent art collection and educational institution was looking to undertake an ambitious new technology-centered project, it had a few unique considerations in mind.

What sets this institution apart—which holds one of the world’s greatest collections of impressionist, post-impressionist, and modern art—is its dedication to using technology to inspire, delight, and educate visitors, as well as the mandated lack of contextual information on the gallery walls (to encourage visitors to focus their attention on the art). 

As part of this focus, its technology and innovation team set out to create a new interpretive guide to its extensive art collection—that works on any smartphone with a web browser. 

But it wasn’t just a matter of building an app that would leverage smartphones’ cameras to pop up information—the content needed to be helpful, concise, and personalized.

To build this, the institution needed a partner with technical expertise and skills in building innovative mobile experiences and seamless user interfaces.

The challenge

From early meetings with the client, one thing was clear to the HFC team: even though the art institution wanted to engage visitors through their smartphones, it was not looking for a mobile application. 

The team didn’t want to ask visitors to download yet another app to their phones—this would create friction in the experience, and deter visitors from actually using their solution to learn more about the art.  

However, they still wanted to present museum visitors with information about the collection in a concise and seamless way, especially because there are no plaques on the walls.

Besides the client’s requirements, there were a few other constraints. Because the biggest consideration in any museum or gallery is the safety of the collection, immersive technology is difficult to execute and can be risky. The client’s museum space in particular has some smaller rooms that have to hold large crowds of people at times, which means the audio guides that are frequently used in many popular museums today would be problematic in those spaces. 

On top of that, the collection’s content is static, meaning visitors will typically stay in the same place for a long time, which creates very predictable bottlenecks in the galleries. This was especially important because of the capacity limits that the team had to consider—the collection gallery can accommodate only 250 visitors at any given time. HFC’s solution would need to distribute content in a lightweight and flexible way that keeps movement in the gallery fluid.

The client chose HFC to help them:

• Ideate and prototype mobile-friendly options for its visitors that take into account the museum’s unique set of physical considerations
• Design a seamless interface that swiftly delivered helpful content in a manner that mitigated visitor bottleneck
• Engineer, test, and launch a digital experience that balances critical knowledge sharing while maintaining the visitor’s immersive in-person experience

HFC’s approach

HFC explored a variety of potential solutions that could provide an app-free experience while still leveraging mobile devices. One of these ideas was to build a prototype of a wearable (such as an Apple Watch), but the team ultimately determined that wearables weren’t the best solution for providing information about the collection in a concise way.

After these explorations, HFC landed on a solution that met all of the client’s requirements. The team would create a mobile-optimized web page that could access users’ smartphone cameras so that when they walked through the small rooms of the museum, they could easily scan a piece of art to learn more about each one. 

Another unique element that HFC integrated was the “room” or collection story, which let visitors know a bit more about why certain pieces were placed in specific rooms and how they connected to other pieces in the space. This way, visitors could better understand the thematic connections between the artwork, enhancing their appreciation of the overall narrative and gaining deeper insights into the curator's vision for each room—all without having to download anything onto their phones.

Because the collection guides had to be translated into multiple languages, HFC leveraged a combination of translations and Google Translate API. 

Post-visit experience

At the end of a visit, users could enter their email addresses to get an “End of Visit” recap with a summary of all the artwork that was scanned during the visit. This would extend visitor engagement and allow them to continue connecting with the art even after they’ve left the galleries. 

Tech stack

• Frontend: HFC used React for the frontend Javascript framework, GSAP (an animation library), Bootstrap CSS (for the UI and CSS library), and ScrollMagic, which provided scroll-based animation for search results.
  
  
• Backend: On the backend, HFC used Active Admin for admin users and ElasticSearch to cache artwork information, which could then be queried when users scanned artwork with their cameras.
  
  
• Other services: The final major piece was Catchoom Image Recognition, which let users point their phone camera at the artwork to pull up more information about each piece. The app would capture a few scans automatically and send it to Catchoom, which matched those scans against its library of art (cached in ElasticSearch).

The result: An engaging, app-free way to learn more about art

Almost two million scans by museum visitors later, this collaboration highlights how technology can be creatively applied to enhance visitor engagement, without compromising the museum experience. 

By rethinking the traditional app experience, HFC helped remove barriers to access and streamline the process for visitors to interact with this remarkable collection of masterpieces. The integration of real-time image recognition, multilingual support, and a personalized post-visit experience not only addressed the client's needs but also helped drive the institution's commitment to innovation, shaping the future of museum engagement.

Looking for ways to enrich an in-person experience and create meaningful connections with your audience using innovative technology? Let’s chat!