Developing an expert system: Much work, many challenges, challenging job. | Copyright iStock 912613902

Artificial Intelligence (AI) is and has been on people’s minds for a long time. Advertising agencies and marketing experts talk about:

  • What does Artificial Intelligence mean for marketing agencies?
  • How Artificial Intelligence Is Revolutionizing the Digital Marketing…

Such titles promise much more than most of the blog or webpage entries deliver.

One of the criticisms of AI is that such systems are unable to ace an eighth-grade science exam. The main reason being that current AI systems:

„…[cannot go] beyond surface text to a deeper understanding of the meaning underlying each question, then use reasoning to find the appropriate answer.“ (p. 63)

Schoenick, Carissa, Clark, Peter, Tafjord, Oyvind, Turney, Peter, and Etzioni, Oren. (September 2017). Moving beyond the Turing Test with the Allen AI Science Challenge. Commun. ACM 60(9), p. 60-64. DOI:
Check out the video at the bottom of this post !

Read the rest of this blog entry to:

  • define what an expert system is;
  • show why Pinterest’s updates are based on an imperfect AI system;
  • illustrate the challenges of using AI to augment marketing;
  • watch an interesting video about AI and learning science further below; and
  • ask you for your feedback, input and opinions – join the discussion.

This entry is part of our series of posts on AI. To stay tuned and get the latest updates, including on AI and marketing, sign up for our newsletter.

This project is part of our White Paper project for the Competence Circle Technology, Innovation and Management #ccTIM from the German Marketing Association (Deutscher Marketing Verband).

This post continues our discussion entitled, What is marketing automation?

1. Definition of an expert system

In the 1980s, we were all interested in Decision Support System(s) (DSS) and expert systems. The use of AI garnered a lot of interest from the business press.

Using AI became easier, at least in theory, thanks to the rapidly decreasing costs of calculating or doing the arithmetic for ever larger data sets. This made it feasible to use many mathematical operations to gain insights into user and customer behaviour.

At the same time, AI systems represented the risk of amplifying implicit bias contained in the data sets they were trained on. In turn, some systems can make wrongful inferences or judgments about users. Below we attempt to define what an expert system is.

[su_box title=“Defining an expert system“ box_color=“#86bac5″ radius=“9″ class=“aligncenter max-width: 700px“]

An expert system uses specialised knowledge and expertise from a human expert in a particular problem area and converts it into software code. With the help of such code, the expert system can emulate the decision-making ability of a human expert. It allows the system to perform at a level of competence that is better than that of non-expert humans.

Expert systems are part of a general category of computer applications known as artificial intelligence.
Expert systems can be used to diagnose patients, to put together a system that identifies fake news, and so on. Difficulties can arise when interpreting results produced by „black box“ systems whose workings are often hard to analyse.

Edward Feigenbaum is seen as the father of expert systems.

See also definition by Encyclopaedia Britannica.[/su_box]

Of course, in cases where decisions can be clearly defined with one or even many algorithms (i.e. mathematical operations), we expect expert systems, and thus computers, to take over most of the tasks currently done by humans.

For an expert system to work well, two things are paramount:

  1. its rules and algorithms need to work properly, and
  2. the rules and decisions made need to be the right ones.

Hence, expert systems are often downgraded to represent expert support systems, which support humans in making better decisions. We define expert support systems below.

[su_box title=“Defining an expert support system“ box_color=“#86bac5″ radius=“9″ class=“aligncenter max-width: 700px“]

An expert support system helps people solve problems. Like an expert system it allows the system to perform at a level of competence that is better than that of non-expert humans.

For instance, with Legalos, the user of the expert support system enters relevant information. The expert support system then uses this information and generates a template, for example a contract between a company and its cloud services provider. Here, the expert support system can provide the entrepreneur with several types of standard contracts very quickly. In turn, this helps keep a company’s legal costs down.

Another simple online expert support system is provided by Germany’s Federal Ministry of Justice and Consumer Protection. The service asks the user to enter some information pertaining to data processing and privacy measures. Based on this input it then generates a transparent data privacy policy as required by Article 12 of the General Data Protection Regulation (GDPR). This can then be slightly modified to fit the company’s particular circumstances.

See also: Luconi, Fred L., Malone, Thomas, W. & Scott Morten, Michael, S. (December 1984). Expert systems and expert support systems: The next challenge for management. Boston: MIT working paper #122, Slong wp #1630-85. Retrieved 2018-06-12 from


In general, an expert system must acquire knowledge from experts. Such insights are then applied to a large set of probability-based rules to make a decision.

By contrast, an expert support system still requires the human user to weigh some of the factors and then arrive at a decision.

2. Pinterest updates – more noise

Many companies use such technology. For instance, Pinterest and Instagram use similar AI to figure out what Pins you should check out on Pinterest or which Instagramers you should follow. Twitter operates the same way, and so does Facebook (see your newsfeed) or LinkedIn (whom you should connect with).

Recently, I got just such an update (see image below), suggesting that I go and check out 18 pins I should be interested in, based on my board #MCLago.

How on earth did Pinterest's "expert system" decide that these pins are relevant to my #MCLago board?

How on earth did Pinterest’s „expert system“ decide that these pins are relevant to my #MCLago board?

3. When expert systems fail to augment marketing

As you can see in the image above, whatever criteria Pinterest used to determine what pins might be of interest to me, ‚common sense‘ was not programmed into this decision-making process. How it concluded that I wanted to meet single men is a mystery to me.

Why I should care about Lipitor – a prescription drug – is unclear. Yes, I do post medical stuff, but primarily about minimally-invasive endoluminal or endolumenal surgery, because of my work with Lumendi Ltd.

On the upper left in the above screenshot you can see some people in a photo. The program concluded this from one of my recent pins. I had recently posted something – with video – about a Syrian refugee (the picture shows the trainee with her co-workers and bosses). So the thought was I would like another one. Well, here a deeper understanding of the meaning underlying the item I pinned would have allowed Pinterest’s expert system to find a picture in a similar realm.

Instead, it inferred that I would be interested in „Who’s In and Who’s Out for the Next Season of Nashville„. Seems a little ridiculous.

Basically, an expert system needs to be able to do more than do simple math. Moreover, predictions are not enough to automate the decision-making process or task with the help of AI (see Agrawal, Gans & Goldfarb, Spring 2017). Below, we list the six key things an expert system must be able to handle to get AI to deliver the most value.

Agrawal, Aja, Gans, Joshua S. & Goldfarb, Avi (Spring 2017). What to expect from artificial intelligence. MIT Sloan Management Review, 58(3), pp. 23-26. Retrieved 2018-06-12 from

[su_box title=“Expert system for automation: Performing these 6 actions is a must“ box_color=“#86bac5″ title_color=“#ffffff“ radius=“5″ width=“px 700″ ]

An expert system not only executes tasks efficiently, but more importantly, gets a few things right, such as:

1. Data analysis: What kind of photos or status updates does this individual post?

2. Prediction: What action would the recipient take and / or would this potentially be of interest to the customer?

3. Judgment: Yes, this status update / photo is of interest to the user / customer.

4. Action: Include photos of interest and mail out newsletter to subscriber, user or customer.

5. Key Performance Indicator (KPI): The recipient has clicked on several of those 18 suggested pins. This expert system did better than average.

6. Quality of service: The pins the client clicked on provided content that represents added value for this user.


Unless the expert system we use can do the above, marketing activities are more likely hampered than augmented.

4. Ultimate test: Does this content answer the question I am asking?

As pointed out above, whether the user clicked on several suggested pins is one possible KPI. For instance, I clicked on more pins than could be expected. Nevertheless, ultimately it is not the clicks on pins recommended to me by Pinterest that matter. Instead, the ultimate criteria for a user is whether those pins provide information that represents added value.

In my case, that did not apply. To illustrate, I checked out the pin about 10 KPIs in marketing, which brought me to a blog entry (see image below).

When an expert system cannot deliver quality: Pinterest recommends pins that mean little or nothing to me.

When an expert system cannot deliver quality: Pinterest recommends pins that mean little or nothing to me.

As the above shows, somebody is spreading her opinions regarding KPIs. We all know that the life cycle of a client is important, but if you are running a start-up, this could be of lesser importance than getting new clients who can help you pay the rent.

Strategising your sales revenue approach is interesting, but not something that everybody needs to do. Treating your clients respectfully and providing a service that they feel is worth the money they paid you most certainly helps. When it comes to revenues, that applies regardless if you track it with a spreadsheet or do it on a piece of paper.

5. What is your opinion?

The verdict is simple. The expert system that Pinterest uses to serve me weekly or more with an email of suggested pins does not do a good job. The recommendations it makes indicate that the AI system lacks a deeper understanding of the meaning underlying each pin I uploaded. In turn, it cannot source pins that might interest me.

But do not be fooled, neither Twitter nor Instagram do better with these things. Developing a well-functioning expert system takes a lot of work and testing.

However, the fact that expert systems do make errors was already pointed out by researchers in the 1990s:

Williams, Joseph (1990). When expert systems are wrong. In Proceedings of the 1990 ACM SIGBDP conference on Trends and directions in expert systems (SIGBDP ’90), p 661-690. DOI:

On reviewing the challenges and benefits of expert systems and neural networks, things do not appear to have become easier in 2014, even though the benefits can be substantial (e.g.,

What I would love to know is what you think about these issues in 2018 (#ccTIM will continue updating you on this subject):

  • Do you think AI (artificial intelligence) will revolutionise marketing? Please explain why or why not.
  • Do you have examples of great expert systems, for instance in marketing, management or production?

The author declares that he had no conflict of interest with respect to the content, authorship or publication of this blog entry (i.e. to the best of my knowledge, I got neither a freebie from any of the aforementioned companies, nor are they our clients).

Check out this video, worth watching – see quote at the beginning for reference to research paper that is source for video below.

Augmented marketing services can make our lives easier but automation can result in undesirable outcomes | Copyright: Death to Stock Photo, 2200

In short, is ordering your pizza online a result of the restaurant augmenting or just automating the sales process?
Structured, rule-based processes can be performed by robots.
Complex unstructured marketing processes that may result in different or unique outcomes are difficult to automate.
To illustrate, bento lunch boxes in Japan contain little portions of rice, fish, meat, pickles and other delicacies packed in plastic box. These items are extremely hard to grasp, thus automation is a challenge.
Here are the three answers marketers must answer soon to stay ahead of the pack.

♥ Curious? Join 1,500+ other subscribers to this blog’s newsletter and read on!

This article is part of our series of posts:

  • What is …. more follows soon

1. What is automation?

Doswell made a distinction in 1983 in his book „Office Automation“ between tool and machine. In its basic form, a pencil is a tool. It cannot be used without the human writing or drawing with it. A typewriter is a machine that requires a human to use it, even if it is electric: a IBM Selectric typewriter.

Doswell defined a word processor as a programmable machine. The fact that it can function in part without a human working it results in automation.

In 1992, I wrote that the effective use of technology requires adjustments in an organisation’s structure, processes and workflows.

Today, we talk about automation, while we actually mean augmentation of services or tasks with the help of software code. We code certain steps or decisions that may result in algorithms that perform routine, rules-based processes. Of critical importance is that the outcome results in one single correct answer (i.e. a deterministic outcome).

But this situation often fails to apply. If I have a book’s ISBN number, I can soon tell my device at home to order this at my favorite bookseller. However, as soon as I enter an incorrect number, the outcome is no longer clear cut. For instance, the system could suggest the correct number, or else show me the name of the most likely author, title and so forth and I can decide if the book I want is listed.

Interesting readDoswell, Andrew (1983). Office automation (see p. 123). Chichester, UK: John Wiley & Sons.

Takeaway: Automation is a not a dichotomy but a continuum

Our understanding of what automation means is a constantly moving target.

In early 2016, Facebook used human editors to develop trending news lists. Some claimed these editors introduced a political bias. In turn, Facebook decided to automate this task and left the job to algorithms.

Later in the year Facebook discovered that algorithms cannot tell a real story from a hoax (i.e. fake news)… Facebook trended fake news.

2. Augmenting marketing services

The above illustrates that automating services is not easy. More often than not, only single tasks end up getting automated. Once things get complex, or several outcomes are possible, automation is very difficult to accomplish.

Automatic marketing services could fail either in unusual situations or in ways that produce unusual situations. The latter may be a tricky situation where various skillful responses may prevent a total disaster or only one specific response can, such as a pilot preventing a plane crash or a powerplant operator preventing a nuclear disaster like Fukushima or Chernobyl (see Tim Harford, 2016-10-11).

Neil Patel and Ritika Puri offer this definition of marketing automation:

[It] connects multiple touch points and marketing channels including social media, email marketing, and content marketing…
Marketing automation makes it easier to send personalized, 1:1 targeted messages. In other words, [it] makes communication stronger…
…Different marketing automation platforms are designed for different types of businesses… Act-On… comes with email, website visitor tracking, lead management, social media, CRM, reporting and analytics. A core value proposition is that business owners can execute their marketing from one place to (1) generate high quality leads and (2) transform those leads into sales…

The above definitions are all important, but they describe mechanisation of marketing processes. The results are what is called single outcome situations or deterministic outcomes.

For instance, you subscribe to a newsletter and get an opt-in email. In turn, you click on the link provided to confirm your signing up for the newsletter. This in turn triggers a thank-you note and so forth. Thereafter you are on the subscriber list and will receive the next published newsletter.

When using a chat box on a website, the operator or system may provide you with standardised replies for those questions that were previously listed in an FAQ (Frequently Asked Questions).

But instead of following these various definitions, it is better to focus on the characteristics of the marketing services that these tools are supposed to automate.

Takeaway: Augmentation still beats automation

By using machine learning we can improve industrial processes, marketing and customer experience.

Augmenting marketing processes is a far more common way to leverage technology than automation.

Interesting read: Lacity, M. C. & Willcocks, Leslie P. (Fall 2016). A new approach to automating services. MIT Sloan Management Review, Vol. 58(1), pp. 41-49. Retrieved 2017-07-31 from

Marketing automation at its best? "When the Going Gets Tough, the Tough Go Shopping" | Copyright: Death to Stock

Marketing automation at its best? „When the Going Gets Tough, the Tough Go Shopping“ | Copyright: Death to Stock

3. The next five years in marketing automation

So where will we be five years down the road? How will automation of marketing services look?

Today, it is primarily defined using simple deterministic processes to define the term, such as:

Marketing automation involves a software platform that can be used to deliver content based on specific rules set by users.

Järvinen, Joel & Taiminen, Heini (2016). Harnessing marketing automation for B2B content marketing. Industrial Marketing Management, 54, 164-175. doi:10.1016/j.indmarman.2015.07.002. Retrieved 2017-08-08 from

Interesting read: National Science and Technology Council. Networking and Information Technology. Research and Development Subcommittee (October 13, 2016). The national artificial intelligence research and development strategic plan. Retrieved 2017-08-07 from

This is the beginning. Algorithms in marketing may be alluring, but as Spotify’s music recommendations illustrate to us users, algorithms do not necessarily have our best interests at heart. They optimise things to enable the company to sell you more music – and so, make more money (see Brian Whitmann, former principal scientist at Spotify, December 11, 2012).

Takeaway: Manage your risk

Algorithms will get better but how much things will change nobody knows. Amazon’s two divisions – advertising and cloud-computing – have relied on self-service portals to attract new clients. New sales staff hires at these Amazon divisions in the second quarter of 2017 are the primary reason for the increase in the company’s headcount (from 351,000 to 382,400 at the start of quarter three).

Larger companies want personalised service. Their needs are too complex to be met by a self-service portal Amazon offers. The result is that staff has been hired to service large firms. When things get too complex, augmentation of services is the first step but automation is not always feasible.

[su_box title=“Marketing automation vs. marketing augmentation: 3 critical questions to be answered“ box_color=“#86bac5″ title_color=“#ffffff“ radius=“5″ width=“px 700″ ]

1.  Is there a single correct answer?

Yes, the customer wants to download a white paper or a checklist.

Or, the individual passes a security check and may therefore enter the secure area, etc.

2.  Can we develop a set of likely answers?

In many situations there could be more than one answer as outlined in point 1 above (the ISBN – maybe the client switched two numbers). This means the answer is a no, but what if…

For instance, based on the user’s past order history, the system knows that this could be a book addressing marketing issues. Going through the database, two options come up where everything is correct as typed in by the client, except for the two switched numbers in the ISBN.

So the system responds in some way and shows the person the two possibilities including author, summary of book and cover. The client can then say yes to one or no to both options of books as presented by the system. A ‚No‘ answer could then trigger two or three more options, and so on, hopefully resulting in the person ordering one or two products.

3.  What degree of agility is required for performing / automating this marketing task?

One outcome may be a less than optimal recommendation list of possible songs on a client’s playlist.

In the case of advertising business, adding sales teams will help Amazon attract bigger clients. The latter do not appreciate the self-service currently offered; their service demands are too complex, requiring humans to consult.

As factory robotics has taught us, robots lack the agility of humans. It makes little sense to have a self-driving truck, if it needs a driver to unload the contents with a forklift on arrival.

If we have a box with 3 products, pens, pencils, fountain pens, the robot needs to distinguish between them. Until robots are able to do such work, Amazon will continue to use humans to pick up products in its warehouses. [/su_box]

Getting things organised - marketing autmentation | Copyright: Death to Stock Photo Workshop 8

Getting things organised – marketing autmentation | Copyright: Death to Stock Photo Workshop 8

4. Have your say – join the conversation

Organisations can use marketing automation to generate multiple business benefits. Cost savings, better customer experience and better quality can be the result of such work.

But where agility is needed, automation becomes a challenge. Unless we can train robots in an intuitive way, rather than program each possible step, move and outcome (if x then do A or ask for B…), marketing automation is more likely to be marketing augmentation.

[su_box title=“What is marketing automation? THE definition“ box_color=“#86bac5″ title_color=“#ffffff“ radius=“5″ width=“px 700″ ]

Sales focuses on the seller’s need to convert the product into cash.

Marketing focuses on satisfying the needs or solving the problems the client may have.

Content, affiliate, dialogue, permission and other marketing activities are just different parts of what marketing entails.

In turn, marketing automation tries to adjust an organisation’s structure, processes and workflows to optimise various activities such as content, affiliate, dialogue and permission marketing.

Going down memory lane, let us not forget

However, mechanising part of marketing does not mean with have done a terrific job.
Just think about direct marketing and software companies like Hubspot and the unnecessary things you receive daily in your letter mailbox or e-mail in-box.

As soon as things get complex, automation becomes tricky

In the early 1980s we talked about office automation. Today we have ever more people working at some kind of office. Even though things have been digitised we use as much paper as then to print…. and many things still need to be optimised further to reach an automation level that allows us to send staff to the beach.
We are still ways off having humanoid robots stand at our office door to take on our tasks.[/su_box]

But what do you think?

Source: What is marketing automation?

What is your opinion?

  • Do you think AI (artificial intelligence) will revolutionise marketing automation?
  • Do you have examples of great marketing automation with the possibility of multiple outcomes (probabilities)?

The author declares that he had no conflict of interest with respect to the content, authorship or publication of this blog entry (i.e. I neither got a freebie from any of the mentioned companies nor are they our clients to the best of my knowledge).