The Essential Role of Hands-On Experimentation in Achieving AI Success

During a recent talk at the University of Wisconsin-Madison, I urged students to broaden their understanding of AI, moving beyond typical uses such as for studying or social media engagement. I initiated the conversation with a provocative inquiry: Is it possible for the effects of AI to be both exaggerated and underestimated at the same time?

I believe the answer is affirmative; AI can indeed be both over- and under-hyped. This assertion is rooted in a crucial observation: our perceptions of AI are heavily influenced by the amount of time we dedicate to independent (DIY) training and experimentation with the technology.

Experimentation with emerging technologies is vital, and it’s especially critical when it comes to AI. Reflecting on my own experience, I wondered if my own utilization of AI was too restrained. Consequently, I resolved to commit additional time to DIY experiments.

Tinkering with AI and similar technologies can dramatically alter your perspective and assist in overcoming the “trough of disillusionment” that often accompanies overhyped innovations, while simultaneously creating more value for your clients.

Explore further: Implementing AI strategies for your marketing team

Recall Your ChatGPT Experience

The last couple of years have certainly given many of us a sense of AI vertigo. Personally, I was in a fortunate position at the time of ChatGPT’s release.

In late 2022, I decided to step back from my corporate leadership responsibilities for the longest break I would take. I eventually pivoted into consulting and education, unaware that this hiatus would align perfectly with ChatGPT’s unveiling. It’s still intriguing to hear how OpenAI referred to it as a “low-key research preview.”

This break allowed me ample opportunity for self-guided training, testing, and learning. Many of my friends and associates who remained in corporate roles lacked the time for exploration, leading them to quickly label AI as overrated.

Those colleagues often lacked the support or authority to carve out time for personal training. I was fortunate enough to attend live events instead of just relying on recorded sessions. I immersed myself in related podcasts, but most significantly, I dedicated time to hands-on experimentation, which reassured me of AI’s worth, notwithstanding its challenges.

Navigating the AI DIY Chasm

To emphasize the significance of DIY time, I created a personalized hype cycle diagram reflecting various training triggers we all encounter.

Gap 1 represents the duration between the launch of a technology and the moment you initiate learning about it, often prompted by an external training trigger, perhaps through your job or a connection network.

Gap 2 is marked not only by the utilization of the technology, but more crucially, crossing what I refer to as the “DIY Chasm.” This involves deftly navigating limitations to reach those enlightening “A-Ha” moments on your own terms.

The total time spent between Gap 1 and Gap 2 significantly influences the productivity and value one derives from AI in relation to its capabilities.

In my previous jobs, I might have shared the belief that AI was over-hyped. The daily demands of managing projects had ingrained certain approaches to martech in me. When encountering challenges, one could usually find a quick solution online. However, if that information wasn’t applicable to your specific situation, progress could stall, and it was easy to blame the software.

Because generative AI isn’t restricted by those original boundaries, further experimentation is even more essential to maximize your results.

Fast forward two years, and I have faced the challenges of Gap 3. My role in education and consulting limits my time for DIY experimentation, while the pace of AI development continues to accelerate. Some days I feel ahead, but every time I dedicate more time to testing, I experience those “wow” moments.

It’s evident that the real constraint lies with me, not the AI technology itself. Keeping up with thought leaders who remind us that we’re working with the “least advanced AI” right now is incredibly beneficial.

Increased Content: Is It Truly Enhanced?

I often credit Scott Brinker with shaping my perspective on “citizen martech.” I was eager to explore the new MarTech for 2025 report authored by Brinker and Frans Riemersma. However, due to my time limitations, I could only skim the report briefly, not delving as deeply as I would like. (You can find summaries and download options on the MarTech website.)

A quick glance at the findings indicated that the primary applications of AI focus on content and personalization throughout various stages of ideation and distribution; the sheer number of such mentions suggests that AI’s impact is not, in fact, exaggerated.

Nevertheless, it’s still uncertain whether customers are noticing these advancements. My latest DIY AI explorations yielded insights for future personalization efforts.

Explore further: Customers are underwhelmed by AI experiences

Tailored Content ‘Channels’

I fully intend to return to Brinker and Riemersma’s comprehensive 100-page report. However, I’d like to highlight two AI-enhanced shortcuts that have helped me bridge the value gap while indicating the potential for more personalized content.

First, prior to a recent trip, I uploaded Brinker’s chiefmartec articles into ElevenLab’s Reader. The AI-generated voice read the articles to me. This experience was surprisingly under-hyped, revolutionizing how I consume long-form content.

A glimpse into ElevenLab’s Reader app.

Next, I uploaded the complete report into Google’s NotebookLM, allowing me to listen to the AI-generated audio overview it provided.

This audio overview creates an engaging “podcast” with two lively hosts discussing key themes from the chosen source material. It enables document uploads, YouTube links, web pages, and more.

I also generated a NotebookLM AI overview for my latest series of articles for MarTech.

The strength of NotebookLM lies in its “source grounding,” capitalizing on the overall model (Gemini) within this multi-modal framework. As Steve Johnson, a NotebookLM co-creator, stated on the Google Deep Mind Podcast, it serves as “a type of personalized AI that excels in the information that matters to you.”

If you’re interested in NotebookLM, I encourage you to explore beyond its audio overview feature. It allowed me to engage in a “conversation” with Brinker and Riemersma through my interactions with the AI overview.

Screenshot showcasing NotebookLM responding to an inquiry using source material.

At this juncture, it became evident that I must integrate a crucial element into my learning agenda for 2025 — the content layer. As highlighted by Rasmus Houlind, this entails a series of interconnected content LLMs collaborating within a martech framework to enhance personalization.

I believe Houlind would support my proposal to infuse a customer tone derived from previous discussions, including emails and meeting notes, which I discussed in Part 2 of my series.

Our Pathway to Personalization

This journey towards personalized content discovery encapsulates the overarching trends I’ve addressed in 2024. The return on investment from crossing the initial DIY Chasm two years ago with genAI has proven beneficial, but its longevity is diminishing as we navigate through AI vertigo. I must allocate even more time for transcending the DIY Chasm than ever before.

With the assistance of the personal AI technologies I’ve described, I was still able to extract valuable insights, despite a compressed schedule. The integration of Q&A, AI interaction, and my preferred audio formats has helped tailor the key insights from the content to my own preferences.

Now, it’s imperative that we prioritize scaling these strategies for our clients.