PegaWorld iNspire 2023: Panel - So You’ve Invested In Next Best Action… Now What?

Investing in a next-best-action approach can be incredibly valuable for both your customers and your business. But once you’ve committed, what’s next? How do you set up your people, processes, and programs for success ­– and ensure you’re getting the value expected from that investment? Join Pega’s Mark Davies as he interviews next-best-action practitioners from three of the world’s most innovative enterprises to showcase their best practices for socialization, centralization, rapid delivery, measurement, change management, and value capture.

Transcript:

- Thank you, everyone, for coming to this session. It's fantastic to see you. A good turnout. I think this is a topic that is interesting for lots of people. We're gonna spend the next 35 minutes or so discussing how some of our most innovative clients have driven phenomenal success with a customer-centric approach to customer engagement. Becoming customer-centric is as much a change in approach or philosophy as it is in technology. All organizations wanna see massive improvements in customer value, customer engagement, but in order to achieve this, they need to more than implement EAI, get the right data, put in place the integrations, and put in place an operating model. They need to think a bit differently about how they go to market, how they change from being focused on products to being an organization that focuses on what their customer wants. So today, I'm gonna explore this topic with three experts from our client community, individuals that have managed to bring their organizations along on this journey so that you can benefit from their experience. So I'm gonna start by introducing myself. My name is Mark Davis. I work for Pega and I report to Rob Walker, who you may have seen on the main stage this morning. And I lead a team called the Business Excellence team. We're focused on driving success for our one-to-one customer. I've been working in the kinda one-to-one customer engagement space for almost 15 years at Pega. Prior to that, I was a customer at Pega where we implemented one-to-one customer engagement for a telecoms company called Orange, a company in the UK. So I've had, you know, a lot of time working in this space, seen a lot of different examples around the world. I've been fortunate to work with client organizations in multiple geographies, multiple industries. So it's been a really great learning experience over the last 15 years. So I'd like to start by asking each of my three guests to provide a bit of an overview on their organization, the challenges they face in their markets and their role and background. Maybe Suzanne, we could start with you.

- Sure, thanks, Mark. Okay, so good morning, everybody. So, Navy Federal Credit Union is the largest credit union in the world, and it's over 90 years old. And we serve all branches of the military in the United States and their families and veterans. And our mission is very simple. It's to enhance the financial wellness of our members. So I lead our Orchestration & Analytics team, which is responsible for defining and building all of our Next Best Action, delivering analytics on their performance, and then also launching all new channels powered by customer .

- Cool. Thank you, Suzanne. So maybe Joe, we could go to you next.

- Super. Thank you, Mark. Good morning, everyone. So I'm responsible for customer analytics, marketing transformation, and marketing operations as part of the Bank of Ireland Group. And we sit in the central marketing function within the group. And just to break out maybe the Customer Analytics team as it might be relevant to the conversation as we go along. There's four sub-teams there. We've got our commercial insights team that sits as part of our cross-functional marketing squads, our data science team that creates our propensity models, our decisioning practice, and our marketing effectiveness team that measures everything that we do. And in terms of Bank of Ireland, and again, similar to Suzanne, we put financial wellbeing at the heart of everything we do. Bank of Ireland is a multi-geography company. We operate on the island of Ireland as a full-service traditional pillar bank. All of the usual products and services that you'd expect from checking accounts to loans, to mortgages, to business banking, wealth and insurance. We have operations in the UK where we operate direct with some niche products and we operate indirect via our partners in the post office and the automobile association as well. We have a corporate banking arm that's service across a number of international countries. Just in terms of the marketplace, Mark, just to finish that up, the Irish marketplace is a very vibrant one. We have one of the youngest populations in Europe, one of the fastest growing economies. Having said that, we've just seen the withdrawal of two traditional banks. They've withdrawn back to their home countries, KBC and Ulster Bank. That's given us a once in a lifetime market share gain opportunity. And we do quite a vibrant FinTech landscape as well with competitors in niche product areas.

- Cool. Thank you, Joe. And maybe Jeroen, we could get some details on what you do and what you do at Achmea.

- Yeah, good morning, everybody. I'm Jeroen I'm from Holland and I work at an insurance company, the biggest insurance company of the Netherlands actually, Achmea. And we have 10 million customers and three power brands within our company. We have Zilveren Kruis is our health insurance brand and we have Interpolis that is the insurance brand of the Rabobank and Centraal Beheer, that's the direct, right, that I work for. And it is a non-life and banking company of insurances. And I personally, I'm the head of the Omnichannel Personalization Department and we are responsible for making every interaction with our customers personal and relevant, and consistent of course. And yeah, looking forward to this.

- Great, well, thank you for that, everyone, for those backgrounds and the details, and the companies you work for. Maybe you could spend 20 seconds or so taking us through what your solution does, for example, the use cases, the channels, and the benefits that you've seen. And maybe Suzanne, we could start with you again.

- Sure, yeah. So in the first two years since we launched, we now have six channels powered by Next Best Action. So we have email authenticated web, paid social media, direct mail, contact center branch. And you know, it's been exciting to see that journey and a lot has happened. I mean, definitely Pega customer decision have enables us to move very quickly. And so yeah, we've been seeing some very good success as a result of being able to orchestrate and arbitrate member communications across all those channels.

- Cool. Thank you, Suzanne. And maybe Joe, I can ask you the same question.

- Yeah, so we've been live since January last year, a little bit less than Suzanne, so about a year and a half into our journey. We went live in Pega Cloud with version 8.5 on two channels, inbound authenticated mobile app and desktop app. We focused on service NBAs to start off uniquely on service NBAs just to allow us to learn the product and not have the pressure of sales coming at us. We upgraded our data during the year. Last year, we added more data to able to differentiate our personal customers from business and north of Ireland from south of Ireland. And then we upgraded from 8.5 to 8.8 at the start of this year as well, so we're up to 8.8 now.

- Cool, and what about the use cases that you have in place?

- Use cases? We focused almost entirely in the first instance on service use cases. And we've seen some super results from that. Things like data quality, maybe we'll talk about 'cause down in the contact center, a self-service up. So by focusing on those use cases, we've seen a lot of success coming from those as well as the sales one in the latter half of last year in the start of this.

- Fantastic. And maybe, Jeroen, we could ask you same question as well.

- Yeah, we started in 2015 with decisioning with-

- A long time ago

- Next Best Action.

- Yeah, it was called then and we haven't been using a double time and it connected all the channels there at this point. Currently, we're on H7, CDH-H7 and migrated to the cloud this year. And in this eight years, we started out small in the inbound channels on the website with commercial use cases, retention use cases, and at this point, we are also using service use cases as well. So for all channels and all interactions, and all types of actions and cases.

- Cool, and I think when we were prepping for this, you were talking about how you probably deliver about a million impressions a day.

- Yeah.

- To your NBA solution.

- A million overall channels.

- Yeah.

- A million a day. Yeah, at this point.

- Fantastic.

- So yeah.

- Yeah. Okay, great. So maybe we can get into a bit of detail now. Thank you for setting the scene everybody. What I'd like to do is start by discussing the organizational changes that you've put in place to operate the capability. So again, Suzanne, I keep picking on you to go first, I don't know, probably not fair. Maybe, we could start with you. How does the business as usual process work at Navy Federal?

- Sure, so we use Agile methodology to deliver not only new capabilities, but also new Next Best Action. And that's been critical to our success because there are dozens across the Navy Federal Organization involved in delivering on this effort. And so every 10 weeks, we hold a SAFe Agile ceremony called Program Increment or PI plan. And that's where we align on what the priorities are, what the dependencies are across all of our business units, we called value streams. We group to value streams, and we look at the needs of the business, need for speed to market, the need for ensuring that we are balancing our resource constraints to ensure that we all know what we're gonna be delivering across those 10 weeks. And then in addition to that, it's very well understood across the entire marketing organization, across all of our business unit partners, what we're going to be delivering every two weeks. So every two weeks we've got dozens of new Next Best Actions that are launching into production.

- Fantastic, it's really interesting to hear how you operate that. Jeroen, how does Achmea operate, the same question to you.

- We also have the SAFe Agile structure within our company. And specifically for personalization. We have five Agile teams in place, three business teams, two teams that built the actions and the treatments. And we have two releases every week with the new optimizations, new actions, and all the new integrations. And we have the team consists of personalization marketers, that is the name we give our decisioning architects within the team. So they're the personalization consultants, but also decisioning architects that built the logic in the CDH application. And the third business team is analytics team. It consists of the data scientists, the data engineers, and the data consultants that built the customer view, maintain the models we have within and outside of CDH and the analyst, they built the reports we use to analyze and report on the personalization efforts. And we have two IT teams currently. And I was mentioning to you that the second part of this year, we are moving away from centralized decisioning teams. So we're putting the personalization marketers within the marketing teams. Actually, that's due to the fact that we moved to the CDH-H7 application. There is a NBA designer available, but we can standardize the work and hopefully it it'll help to keep the performance up to speed but we're looking forward to that change, yeah, actually.

- I'm sure it'll help. I think also when we were prepping for this, you mentioned that in a typical release, you'll get around 20 actions to market. That's twice a week-

- Yeah.

- You're putting about 20-

- Twice a week, 20 optimization, it can be actions, treatments, or all types of changes.

- Okay, cool, so...

- Yeah.

- Good volume change.

- A lot of changes-

- Yeah.

- And an automated release of cycle.

- Okay, so maybe we could change gears a bit and I'm gonna pick on you again, Suzanne, I'm afraid. It was really interesting to get the insights, the way you operate, your processes from both of you and the way you've structured your teams to support that. But I'd like to dig into a bit more detail on a couple of specific things on Navy Federal. How do you manage the balance between the rapid and frequent delivery of actions, right, alongside the slower cadence of implementation of new tech, of new capabilities?

- Absolutely, so we have... I already mentioned the business as usual, Scrum team. They're the ones that define and build and launch our Next Best Actions. We have two other Scrum teams, one of the other Scrum team is responsible for all of our channel integrations. And then a third Scrum team is responsible for enhanced and new capability development as well as any production needs. And so how do we manage, you know, the dependencies, how do we manage the ability for all of those teams to work seamlessly with one another? And Mark, as you mentioned, right, the business as usual, Scrum team is moving very quickly. The Scrum teams that are focused on channel integrations or new capability development, they move a little bit more slowly. So the business as usual team is gonna take priority, but when they're not using, say, the integration environment or the performance test environment, then the other two Scrum teams then work collaboratively to determine when they can use those environments. In addition to that, we hold a weekly scrum of scrum. We also meet weekly with our system architects to make sure that we're aligning and in sync on priorities. And then in addition to that, we have, you know, of course, every Scrum team has a product owner. And so we meet on a very regular basis at least once a week. And we're revisiting our epic and feature roadmap at least once a month. And as new priorities come up, we'll shift what those epics or features out in order to bring others in.

- That's great, I mean, you're being very responsive to the business needs and managing to balance like the way you balance the slow cadence of technical change with the fast cadence of business as usual, getting actions to market. So Joe, maybe we can move on to Bank of Ireland and talk about a slightly different topic. Stakeholder engagement is really, really key. Who are your internal clients within your organization?

- We've got, like, a variety of different clients that we're managing all the time and similar to some of the areas that Suzanne's called out there. So we're still in build, we're still adding more channels, we'll add outbound digital this year. So we've got a lot of stakeholders in that, IT change community. But on the run side of the house, just to focus on that, probably two key stakeholder groups for us. First of all, our marketing squads, those Agile cross-functional marketing squads I mentioned earlier on, they're a key stakeholder group because first of all, part of my team sits in them. Second of all, they're relying fully to the business. So the business is split into value streams, home buying, everyday banking, wealth and insurance and so on. And there's a squad per value stream. So they're driving, you know, interfacing with the business, driving demand into us. At the same time, the second stakeholder group for us would be the business themselves because we see ourselves in analytics as being part educator, part provider. So you've got this, you know, need to bring data, and analytics, and insights to the business, which is often the thing that drives that demand down into the marketing squads-

- Okay.

- But there are other insights that we bring to them that don't drive marketing demand. So those are the two key groups for us.

- Yeah, and that sounds like a fairly typical setup with different groups responsible for different aspects of engaging with the customer. Could you talk a little about their objectives, the things that those different groups are measured on?

- Yeah, I mean, objectives in terms of... We would summarize the KPIs into, you know, revenue up, cost down NPS up, I mean, quite simply and it's consistent across the different value streams. The couple of differences to, you know, everyone wants more leads, more applications, more draw downs, and we measure across of that. In our business banking area, we would also measure referrals because we have on the ground sales teams where that's important in our everyday banking. I think I mentioned at the start, we measure data quality, how we're using Next Best Actions to drive up data quality, how we're driving calls down at the contact center and how we're driving up self service so that would encapsulate all of those KPIs and all that.

- Yeah, and I think when we were talking about this before, you were talking about getting better at using some of the tools to help you manage that.

- Yeah, yeah, look, we actually attended a session yesterday with Philip Monahan, it was great to understand the suite of tools that Pega have out the box. And we're like so many other clients of yours that we speak to, we feel we're only scratching the surface of, you know, we've deployed a BOE now, value finder, scenario planner, all of those tools, I mean, they're there. We see the benefit from them, we've got a good established base of capability now and we need to start using more of those tools as we go.

- Cool, and it's great to have things like value finder to help you predict outcomes and try and understand how you drive up customer engagement. So Jeroen, maybe I could start with you for this next question. We've heard Joe describe how the key stakeholders within his organization are aligned against specific objectives. I'm assuming that's similar for Achmea. I see this across most of the clients I work with. What did you do to get those stakeholders on board?

- Yeah, stakeholder management was day-to-day business. And luckily I was able to convince our CMO in 2015 that we needed to move over to a centralized decisioning hub to handle our one-to-one engagement. And we started off with a small team then and we're successful with the value-driven use cases. So commercial and retention use cases that delivered value. We use control group to measure them and show that they're valuable. So it really helps to convince other stakeholders in the organization that it's important that you do this for your customers. And it became a company-wide strategy to go to one-to-one engagement with a centralized brain and be consistent over all your channels. So that's key for us. Valuable use cases-

- Mm-hmm.

- And of course, we had other struggles because marketers and at least in our organization, tend to send out volumes, large volumes of emails. And we started out with inbound with our NBAs and we moved over after a couple of years to also outbound always on email marketing, but they still tend to want to send out lots of volumes of emails and that it looks almost like a KPI for them. They send out the volumes, but we convince them now you want to drive the sales and your retention KPIs for the success of your outbound campaigning. So currently, we only have always on email marketing campaigns within. Every night, we have a batch that runs over all our customers and look, is there a relevant outbound use case in place and we will send them out. Of course, contact policies and all are in place prioritization. And we also have an unsubscribed KPI, the unsubscribed KPI shows that the unsubscribed list reduces when you use always on and not stand out, only the volumes.

- That's a really powerful KPI I think to bring up actually-

- Yeah.

- With people who are trying to follow that kind of spray and pray or blast approach that-

- Yeah.

- Markets used to follow. 'Cause if you do that, you just upset your customers and they unsubscribe.

- Exactly.

- So Suzanne, maybe we go back to you. It'd be really good to understand how you engage your stakeholders at Navy Federal. You've got alignment with the one-to-one vision here.

- Yeah, well, it's a multi-dimensional approach. And so one of the first things that we did was, you know, critical to our success is we hired a change management expert that specifically focused on Next Best Action. And we paired her with an enterprise organizational change expert who had experience with driving large scale change at Navy Federal. And so lots of collateral created, lots of, you know, roadmap or road show presentations, you know, delivered lots of communications delivered in multiple communication vehicles, so video, newsletters, intranet pages. But what was really important was an individualized touch. And what I mean by that is we would meet individually with our business unit partners and we would share with them what we'd been working on, we'd be listening to their concerns, we'd be addressing them, and we would also be listening for, you know, typical kind of frequently asked questions, if you will. And so there were a lot of questions, for example, on adaptive modeling, "Hey, what is this thing? It looks a little black boxy to me, help me better understand it." And so, you know, built like lunch and learn presentations to walk them through and help them answer, help them understand, how those different features of Next Best Action worked. The other piece of this too, and kind of hearkening back to Jeroen was saying is that while we've always had this really, you know, great focus on what we call grow actions or product acquisition, we also, with Next Best Action really started to hone in on and focus more on balancing those communication strategies. So bringing more of the membership loyalty, nurture, service, retention types of actions. And what was really key about that is that we could align that with the enterprise transformational vision of greater member centricity. And so helping to kind of bring the two together to say, "Hey, this is how we're gonna help deliver on it." And it helped to kind of bring those stakeholders around to that Next Best Action vision.

- So I love the kind of multimodal approach that you took there with all those different ways of reaching out to your stakeholders to find, I guess, methods that spoke to them and worked for them to help them internalize what you were trying to achieve. Joe, maybe we can come back to you for a second. Were there any other lessons that we can take from your experience at Bank of Ireland?

- There's some good parallels with what we've heard so far. I mean, if I take a chronological approach with when we were in the business case development phase for CDH and because we need the business to sign up to the benefits in order to get approval, we spent a lot of time on that phase working with our partners in Pega and the business owners to help them understand the pre PegaWorld compared to the PegaWorld, what the difference would be in terms of customer experience, in terms of the KPIs and outputs. So that's kind of one big learning for us and big, big reason we got choose those stakeholders. The second one was with our marketing squads. When we were in deployment, we had our deployment for our first two channels going on, but we were working with our second stakeholder group, the cross-functional marketing squads on the op model and making sure that was fit for purpose. And thankfully, we had done some good work in the previous years to ensure it was, but you know, clearly we had to tweak to drop the decision practice into that op model and make those connection points to places like design and copywriting as well. So that was important. Probably, last bit I'll touch on and it was a surprising success for us. We were asked to do an immersion session for the group CEO, a couple of months after deployment. And rather than going in with a series of slides and trying to explain it, we again worked with some of the guys in Pega, Rob McLeod and Martin Radford to skin some of the Pega pre-sales environments to look similar to our customer environments and our colleague environments in the contact center. So we were able to create a demo that was more hands on, more real, that showed a customer logging into this kind of notional desktop app, checking a balance being presented with an NBA, taking an NBA, starting a journey. And in our case, we took a personal lending journey but not completing it. And we moved over and brought the customer service and the contact service, contact center agent experience to life where they were logging into take a customer service, getting the screen pop, accepting the call, going through the service issue, but then taking the opportunity to finish that same lending journey because the NBAs were similar. And you know, we could see at that point the group CEO talking to the group CFO and just understanding it really, really well. We rolled out that immersion session many times over last year and this year to the extent that we've gone recently and bought a 20-user full-blown, like, a demo area that we just do demos in. And for that reason because it makes it easier for the stakeholder groups to understand that when they can touch and feel what we're talking about.

- I love the idea of having internal demos to make it real to your stakeholders. I think nothing shows the reality of the solution more than having a kind-

- Yeah.

- Of tangible thing they can look at. Although, I do remember you've done a lot of presentations that involve restaurants and food, an analogy for one-to-one. So I'm surprised you didn't mention those.

- Yes.

- [Mark] But staying with you for a minute, how did you prove the benefit delivered by the solution to those stakeholders? That's really important too.

- Yeah, yeah, we proved the benefit to the KPIs that I mentioned earlier on. I mean, if you separate the KPIs into results from service versus versus sales or growth on the service side, we saw one of our first NBAs was a change of address NBA where we looked at the data and we said we'd present an NBA if we felt that was an address quality issue and we saw huge success, hundreds of percent of uplift on that particular NBA where we were getting customers to self-serve the dress quality issue. We've, you know, blown that out and used it in other areas 'cause you really can be pinpointed about data quality issues for particular customers or cohorts customers. That was one we had the contact center where we had service NBAs. Fraud is obviously a big issue of financial services. In our suspicious transactions line, we were able to show a 15% reduction in call volume to that line. More importantly, we could see the uplift in the self-service where the page that the NBA was referring to went from 500 hits a day to 19,000 hits a day. So, you know, and that's important 'cause it's not just about calls down and then they might go back up again. It's about sustaining that with customer self-service in a way that's good for the customer. On the sales side, it just to draw maybe a couple of metrics. One of the personal lending NBAs, we saw a 46% uplift in leads and a 22% uplift in drawdowns on that. I mentioned small business referrals on the ground sales teams. We put live some asset finance NBAs for small businesses and we saw nearly 100% uplift in referrals for that line mortgages. I mean, we've had many, but mortgages, fixed rate rollers are obviously an issue on the churn end. We saw over 40% increase in one particular model for fixed rate rollers in reduction in churn. So lots of different ways.

- Those are phenomenal results. I mean, it's amazing that you've driven such a change in behavior in your customers from calling the contact center to driving them to self-serve. That's huge. Could you give us a quick overview of how you used reporting in your business as usual process to give stakeholders confidence?

- Yeah, I mean, we probably look at that from a quant and acquired perspective. The first thing we did from a quant perspective is we created a bespoke set of reports that allowed all of our stakeholders in the marketing community to understand for each NBA what the engagement with that was and really to drive that engagement or that understanding of what's working, what's not, and how to iterate what's not to make it better. So that's kind of on the quant side and it's easy to do those reports, but you need to get to the quality side of that to bring people to use it more and what we did there on the quality side was we have a weekly marketing all hands commercial call and we brought, you know, we leaders and we'd laggards early adopters and laggards in the CDH space in the marketing community. We brought the early adopters onto that call where everyone was listening and they were telling their story about how they were using CDH to drive the success in their metrics on behalf of the business. And then, you know, human nature being what it is and we all have that competitive instinct. The laggards then weren't, you know, longing, coming to the table and wanting to know more and create more NBAs, so...

- I love that idea of creating internal competition to get people to pick it up, right? It's a great way to do that. One final question on this topic. How do you drive insight from the capability? Is there a feedback loop into the business to help them shape new actions?

- Yeah, there is, I mentioned on the four pieces of the customer analytics team. The last one is the marketing effectiveness team that measures everything and then feeds that back into the front end of customer analytics, which is our commercial insights team. And so that goes into the squads and it's used as part of what they do but then as importantly it goes back to the business as well where they start to drive differentiation. So we're closing that loop that way and making sure that what we do next is driven by those.

- Cool, so Suzanne, Joe's given us his experience of what they did to demonstrate benefit to their stakeholders, but obviously, well, that's really helpful by itself is probably not enough to convert your stakeholders. You need to continue to sell the benefits to your internal stakeholders. How did you communicate to the internal stakeholders at Navy Federal?

- Well, so with every new channel launch, we would have a separate communication plan because everything has a different flavor. And I mentioned previously that we would use a multitude of delivery vehicles because different people learn differently. And so one of those vehicles is video, while it's a heavy lift to design and build a video, it's also a very effective communication vehicle. So you know, for example, when we first launched Next Best Action, we did an overview video of what is Next Best Action? And when we were hearing a lot about, I don't understand adaptive modeling, we built a video to explain adaptive modeling and then recently, we just launched our Next Best Action to our contact center and branch channels and we pulled together the leader of marketing, the leader of branch and the leader of contact center, featured them in a video where they were talking to the member service representative community about the benefits of Next Best Action for them. And then in addition to that, lots of presentations. So most recently we just celebrated our two year anniversary, you know, since our first launch. And so I had the opportunity to present all that our great cross-functional, cross-departmental team had accomplished in our first two years together and how far we had come in that maturity curve toward more member-centric communication. And we also talked about what was next in our roadmap. And so, you know, I think all of that, you know, just making sure that with every new channel you know, that you launch, you're always thinking about "Okay, what is the best way to talk about this with our stakeholders?"

- Fantastic, so again, I really like the multiple approaches that you take. I think having a video, whilst it may be heavy lift, is really scalable once you've got it. Although I'm now thinking, having seen Rob's presentation this morning that maybe you could use generative AI and you wouldn't even need to have those people in the video kind of spoof it and make them say whatever you wanted.

- Deepfake?

- Deepfake. That's it.

- Okay.

- Yeah, yeah, exactly. So back to you Joe, and back to your stakeholders at Bank of Ireland. And given the shift in thinking to customer centricity, the new measurements of success, were there any changes that stakeholders had to make to their teams?

- If you take the two stakeholder groups, they called out the business now. I mean, we were able to drop CDH in without any impact on the business, on our business stakeholders and nothing there. And I mentioned earlier on the marketing stakeholders or cross-functional marketing squads, very little. It was just about dropping the decision practice in and making the connection points from a process perspective but minimal, you know, we had a lot of work done in previous years as I said to make sure we're operating in an Agile cross-functional way, so it was an easy job.

- Cool, and that was in some ways to answer was a little bit surprising to me, but it sounds because you already had Agile marketing processes in place, it was a pretty easy lift to put CDH into that. Suzanne, how was your experience at Navy Federal?

- Well, okay, so we purposely did not want to change the way our stakeholders worked, but we did make one very important change and that was, I mentioned previously, the Program Increment or PI planning, so we continue to do this as we bring them into certain times of that two-day PI planning where we're presenting to them our understanding of the priorities across all value streams. And that's critical because they're not just seeing the priorities of their value stream, they're seeing everybody's priorities and so they really get a good appreciation or the importance of prioritization.

- Right, and I guess they see the downsides of them forcing their own things through it for the people that are there to present what impact that's gonna have on them-

- Absolutely.

- So I think that's a really powerful way to get those peers work together.

- It is.

- So it's been really great hearing this information from you guys. Great discussion. But I think we're gonna need to wrap in a short while to leave time for one or two questions. But before we get to that, just a couple of final questions for each of you. So if you could go back in time to the start of this to when you started implementing one-to-one customer engagement, what was the one thing that you didn't do that you wish you'd done? And I'll start with you, Jeroen.

- Yeah, do you have a minute, Mark? No, I'm kidding.

- And if you do.

- What was I saying there? No, we had 2015 when we started the decisioning architects in our team, they built the logic on the canvas in Pega NBA and it was a pretty steep learning curve to learn how to build the logic. And there were so many possibilities and the possibilities are still there, but in the latest versions, there is an application Next Best Action, I know you already mentioned and also the best practice framework really helps new team members get easier and faster up to speed. So that's something that would be nice to have it available-

- Yeah.

- That back then but it's great to have it now.

- Hopefully, it accelerate stuff now.

- Yeah.

- Joe what about you? What about at Bank of Ireland?

- I'm thinking about the answer Mark is probably what we should have done more of slightly different to answer your question but you know, I wish we had spent more time as I said earlier, round up understanding the out-of-the box tools and how to use those earlier. But you know, we were in the early stages and we were learning, so you know, we'll get to that probably more use cases, driving more content, driving more NBAs would be another thing. But you know, we started with 26, was over 200 now. Downstream journeys is probably an area that, you know, we're getting great success with our analytics and with CDH, and you know, we're correcting downstream journeys where they're not operating as well as they could. So that's another area that I wish we had spent more time on input.

- Cool, and Suzanne?

- Well, I think you've all heard about this a lot but I wish we had started on Pega Cloud to make it easier on our operations teams. And then the other thing is I wish we had kept up with the releases more frequently. We did just upgrade to 8.8 earlier this year and cut the batch NBA processing time by more than half. So you know, we are forever more gonna stay current on our releases.

- That's a big change, I mean, cutting the processing time for batch by half, half at first is pretty significant, right? That's fantastic. And then flipping that question around, what was the one thing that you did do that you are convinced was really critical to drive your success? And can we come back to you Suzanne, to start with that one?

- I'm gonna say two things. One I've already mentioned, which is Agile methodology. There is so many people involved in this. You heard that with like the Virgin session this morning that it's just critical to keeping everybody aligned in the planning the execution of all of this effort 'cause it's very complex. And the other piece of it too is that I'm glad that we ensured that people throughout the entire organization knew this was not just a marketing initiative. In fact, I mean, it was anything but... I mean, it was digital, IT, business unit, you know, all of us together collaborating. We share in the work, we also share in the success.

- Cool. Joe?

- I think can I call out too. Actually, I think the first one was Pega Cloud, inbound channels and service NBAs will be the first one. You know, doing those, I think it was critically success and not just because of two of my teams sitting in the room, hired the right resource. I mean, our head of decisioning is brilliant and our PO for that squad, they didn't pay me to say that, but they are here. I think the resource, getting that right to Suzanne's point about people is really critical.

- Okay, and Jeroen? And if it's you for that.

- Yeah, I wanna talk about personalization. Yeah, always mention three critical best factors. The first one is the centralized decisioning of course, but also have a central decisioning team in the organization 'cause you really need to build this capability. And the second one is get also business and IT. One Agile team to get all your channels connected. And the third one is a use case, a driven-backlog because of course, technology is crucial for this, but in the end, the customer goals, your organizational goals are delivered by the use cases, so these...

- Yeah. Well, thank you guys for that discussion. I think we've probably got five minutes left for questions. So if anyone has questions, there's a couple of microphones in the middle of the room. If you could walk up to one of those and ask a question. There are people getting up, I don't know if they're asking questions or not.

- Yeah.

- Oh, we have one here.

- [Audience 1] Okay, great session. How does the creative teams fit into this process? So like the teams are actually doing the copywriting, doing the emails, things like, do they sit in your pods or your squads or do they... You partner with them?

- [Mark] Joe, do you want to answer?

- I'll start if you want to. So for us, when I talk about the cross-functional marketing squads, the brand team or the digital team, they're all members of that squad. So within their team that's aligned back to the value stream. They know what content works and you know, how they bring that to life. We do A/B test out as well. And we do look for ways to take successes from one area. But that's where the creative sits today. I love what I'm seeing in version 23 'cause I think that'll change as we go on and that the skills will be different I think going forward.

- Yeah.

- It will prompt engineerings and like copywriting.

- Yeah. Yeah.

- I'll say ditto. So, you know, sitting within each of our value streams, we have our copywriters and our designers for all of our creative and they just work seamlessly with the rest of the marketing organization to ensure that those are delivered, you know, to the right people through the right channels at the right time.

- How about at Achmea?

- Yeah. Yeah. We have cross-functional teams in our marketing department and the creators are part of that. But we have a handover from these marketing teams to our decisioning team, and we build the logic, and the content into the channel.

- Okay. Any other questions from the audience? But we have one here.

- [Audience 2] Could you talk about the ideation phase a little bit? So Next Best Action doesn't always align with custom offerings. I mean, simple thing like a happy birthday could be an action. So who comes up with these ideas? What is your process of getting these ideation phase? You could talk about that. Thank you.

- [Joe] It creates the ideation for the NBAs, you mean?

- Yeah.

- Again, for us it starts, I talked about that feedback loop for us. It could be any number of different places. It could be an insight from the commercial insights team that's feeding the business, that drives a business need. It could be something, it could be an iteration of an existing NBA, it could be a new product or a change to existing product, but the ideation works within the cross-functional marketing squad. So having that cross-functional nature that's important to get that out. 'Cause our design team sits as part of that as well. So there's a discipline to that ideation for us.

- So I'm gonna say ditto as well. It's interesting how aligned we are. So I would just say that we have within the marketing organization, you know, we have what are called strategists, right? And they work with our business unit partners and together they're ideating. And then the analysts also because they're bringing the analysis of past Next Best Action and how they performed. You know, the three of them all kind of combine their efforts to determine what the next thing is that we should be talking about with our members. That should be, you know, that we think could be relevant to them. But in addition to that, we have another stream of insights that are feeding into all the other work that we're doing. And so I think just bringing all of that together, the insights as well as just the learnings from the past to ideate on what is the new next.

- Jeroen?

- Yeah. Of course at our company as well, marketing teams come up with a lot of ideation. But next step we have personalization workshops because we have a lot of possibilities in all the channels and a lot of marketers are not aware of these. So we initiate these workshops and introduce all the possibilities and they create an ideation stream. So that helps us very much.

- Okay, I think we might have time for one more question if there's anyone else that has one. You shouldn't have said that Philip-

- [Philip] Yeah.

- [Mark] But if you wanna go to the microphone and ask that'd be great. The idea of a plant is you don't admit you're a plant.

- Right?

- [Phillip] Okay. So, I was lying . No, there may be a bit. So I think, Jeroen, you mentioned that the idea that you've got product owners who have metrics to meet and maybe they don't even care, they're just tracking to KPIs. Like, I want this much volume and that you need to convince them that there's another way of doing this, right? How did you convince them without telling them inherently, look, it doesn't make sense that they're like, "I don't care. I know that I get 500,000 emails every month that I don't..." How did you make that?

- Okay, well, have you ever tried to convince a marketer?

- [Phillip] That's what I'm asking. That's what I mean. I just don't know, you put on the spot and it's interesting.

- No, so we have a dashboard and we use control groups to measure our net effectiveness of all the use cases we have in place. So we don't have to send out the huge volumes. We get the same results with our smaller, relevant emails. So of course, in the end, the facts speak for themselves.

- You also measure unsubscribe, right?

- Unsubscribe, yeah.

- I think that was one of the key things that...

- Yeah, I already mentioned that.

- Yeah.

- Yeah.

- That was mentioned. You showed them that the unsubscribe lists we reduce. So when, of course you are... When every time you send out an email to a large volume of crowd, there is always a big part of the crowd that the media email is not relevant for it. So a lot of them will unsubscribe and you don't want that.

- Yeah.

- Yeah. I would encourage everybody to do, when you're first starting out, do an A/B test and have half of, you know, the quote "Campaign go batch and blast" and the other half go NBA and it's the proof in the pudding because we did that for a credit card, you know, campaign and the NBA blew away in terms of applications and bookings, the batch and blast. And that was exactly what we needed in order to help, you know, our business units understand the power of Next Best Action.

- Yeah, we're adding up on digital this year, so we don't have that challenge yet, but there's good learnings for us here as to, because the behaviors are the same-

- Yeah.

- But I like the subscribe rate. We'll definitely use that.

- And maybe I can add one little trick because there's still ask for the big volumes, right? And a lot of brand communication for your organization when it's sent out to all your customers. So we introduced a dynamic newsletter and articles in the newsletter are dynamically and we use adaptive models to use the right action or for the newsletter. And we tell the marketers, "Okay, if you want to send out a relevant content for all your users, add content to this new list newsletter." And it pops up at the right time at the right customer.

- Fantastic. So I think we're gonna have to wrap there. We're actually a couple of minutes over, so thank you for coming to see this. I think we need a round applause for our three panelists.