Understanding Filter Rules Within A Query Pipeline

What Are Query Pipelines?

Understanding what a query pipeline is and how it is used as part of Coveo Cloud is one of the core fundamentals in terms of managing a search.

It's a set of rules and parameters that a search query will be modified against. They're easy to setup from an Administrative perspective using the Coveo Cloud Platform.

They are easy to maintain from a Marketing perspective; adjusting rules within Stop Words, Ranking, Featured Results, etc.

As well, when associated against any one of the Machine Learning models, such as Automatic Relevance Tuning (ART), they're improving the relevancy of index items automatically over time.

Where Do You Set Up A Query Pipeline?

Creating a query pipeline is easy. Log into the Coveo Administration Console and on the left-hand navigation, under Search, select Query Pipelines.

Before you even set up a query pipeline though, you'll notice one is already there waiting for you. Essentially every Coveo organization needs at least one query pipeline which is why there is the default pipeline setup.

You can easily create any number of other pipelines to meet your needs, but for most organizations, the default will be all you need. As we'll show, even after you've set up your pipeline, giving it a specific set of rules to follow, doesn't mean you're limited to that from there on out.

There are a number of ways to build upon it and make it fit depending on the requirements.

What's In A Query Pipeline?

Are query pipelines a set of rules? Yes. Are those rules as simple as 1, 2, 3? Not exactly. Each query pipeline is made up of a smaller sub-set of rules. They can be broken down into the following categories:

• Machine Learning
• Thesaurus
• Result Ranking
• Stop Words
• Ranking Weights
• Triggers
• Filters
• Query Parameters

It might seem confusing, but Coveo has made setting each one of these up to be nearly a point-and-click adventure.

Each is powerful on its own, combined even more so, if you know how to use them to their full potential. For the purposes of this article, we're focusing on the Filters rule.

Why? Because it's likely the first place most developers and administrators will go when building a pipeline.

Filter Rules

Perhaps the most used category of a query pipeline alongside Machine Learning and Result Ranking. It's where you can, programmatically, filter your results to return a specific subset of your full index.

You might do this because not everything should be displayed (though we suggest you look at Extensions to reject items if that's the case). Or it could be that you have several search pages and each one is specific to a particular area.

Your pipeline can have multiple Filter rules, so don't feel pressure to lump it all into a single Advanced Query. In fact, from a readability and maintenance perspective, that's just a bad idea.

Each Filter can use a particular query parameter:

• q (query)
• aq (advanced query)
• cq (constant query)
• dq (disjunction query)
• lq (large query)

Before we have a look at the usefulness of each one and how it works within the pipeline, it's important to understand how a Query Expression is composed.

Each time a user submits a query (q) it's combined into a larger query expression.

((q AND aq) OR dq) AND cq

Reading the above you can see that at the very least a result must satisfy the constant query along with either the basic and advanced query OR the disjunction query.

Basic Query (q)

The Basic Query is that of the user input. Whatever the user enters within the search box. It might be a series of words or, if allowed, could be an expression itself.

Advanced Query (aq)

This is the type of query most often used as part of a filter. It's typically done client-side when a user interacts with the search page, e.g. clicking a Facet.

It should be noted, that you could write your entire filter expression within the advanced query.

If you want to get fancy, this is where you'd use QPL Objects. What are QPL Objects you ask? Well, think of them as variables that you could use within your query expression.

Here you can find an extensive list of QPL Objects and what each one does. For example:

(@title=$query)

In the above example, if used as part of your Advanced Query, only results where the title matches the query perfectly would show up.

Disjunction Query (dq)

This is most often generated by the Advanced Relevancy Tuning (ART) ML model and usually not something you create. E.g. based upon a specific query the ML model recognizes that a specific indexed item should be returned.

It should be noted that Match the query or Match the advanced query are enabled this will not be entered at all.

By default, neither of these are selected as most people who bought Coveo want to use the ML.

Constant Query (cq)

While the (aq) is most often where people store their filter rules, it's quite likely they should have put them within the Constant Query.

This is because it's this rule that must be satisfied for the result to be shown. For example:

(@template='Blog')(@filetype='html')

They are straightforward, non-complex queries and should not be of the nested variety.

If you wonder if your expression is nested, then just look and see if you're using [ ] brackets.

If you are, then it's likely you've setup a nested query. Just something to be aware of.

Large Query (lq)

It's unlikely you've needed to use this expression as it's usually a part of a case deflection strategy when sending a large textual data set in addition to the query.

Using The Advanced Query Client-Side

As mentioned above, while the Advanced Query can be stored within the query pipeline itself, the primary intention is to build it during client side construction of the query itself.

With that in mind, let's look at how you might go about building an (aq). This is all done in JavaScript utilizing event triggers.

So let's take a look at a simple example and then dissect it.

Coveo.$$(document.getElementById("search")).on('buildingQuery',
    function(e, args) {
        var expression = "(@source='PROD')";
        args.queryBuilder.advancedExpression.add(expression);
    });

Let's work our way through the example above. First, is the Event by which the function is called.

We're wanting to add our expression during the part where the query is being built. Understanding the available events within the Coveo's JavaScript Search Framework is key to building powerful and effective queries.

In this case, we're performing the action during the buildingQuery phase. This allows us to attach our expression of defining the source upon which to run the query against, to the Advanced Query.

Secondly, is the args.queryBuilder.advancedExpression. Everything as it relates to Coveo's form submission is stored within the args object. You won't just find the (aq) living there.

You might ask, but shouldn't the above expression be in a Constant Query (cq) in the query pipeline itself? And you'd be right, if it was always the case.

What if we wanted to, depending whether or not the url contained the words "dev", switch the source to the DEV source?

If we had already set the source in the Constant Query, this wouldn't work, as no item would exist in both.

By updating it here, we can switch sources programmatically as show below.

Coveo.$$(document.getElementById("search")).on('buildingQuery',
    function(e, args) {
        var expression = "(@source='PROD')";
        if (window.location.hostname.indexOf('dev') > -1){
            expression = "(@source='DEV')";
        }
        args.queryBuilder.advancedExpression.add(expression);
    });

You're also not limited to simple things like a websites URL. You could modify the Advanced Query based upon the user's query itself.

For example if based upon Reporting we discover users are entering in brackets as part of the query, say they're searching for "Automatic Relevance Tuning (ART)".

We know that we have pages where ART is referenced without brackets. We could do something like the following in the doneBuildingQuery phase.

Coveo.$$(document.getElementById("search")).on('doneBuildingQuery',
function(e, args) {
    // Get the users query
    var originalExpression = args.queryBuilder.expression.parts[0];
    if (typeof originalExpression !== "undefined") {
        var newExpression = originalExpression.replace(/[\)\(']+/g, '');
        args.queryBuilder.expression.parts[0] = originalExpression + " OR " + newExpression;
    }
});

Ultimately how you build your Advanced Query is up to you. The important thing to remember is that by using the Advanced Query, your Query Pipeline is not limited by the rules defined in the Coveo Platform.

Taking full advantaged of the (aq) enables your searches to be even more flexible.

Debugging The Advanced Query

While building the Advanced Query can be as simple or as complicated as you wish it to be, debugging it might seem impossible.

How do you know exactly what is being passed as the Advanced Query when it's fully submitted? The answer is simple.

In your browser, open up the Developer Tools / Inspect area. Click on Network.

In your options, select XHR and then refresh the page. You're looking for an entry titled v2.

This is the form submission of the Coveo query. On the right-hand side, under Headers scroll down to the Form Data and you'll see something that resembles the following.

actionsHistory: [{"name":"Query","time":"\"2021-03-21T17:13:27.769Z\""},{"name":"Query","time":"\"2021-03-21T16:20:32.623Z\""}]
referrer: https://localhost:8080/search
visitorId: 369ef54f-bac7-4fa1-a652-ac402aa8b8a2
isGuestUser: false
aq: ((@source=("DEV"))
locale: en
pipeline: default

Through here, you can see your (aq) being submitted and debug any issues, malformations, etc. that may arise.

Understanding each part of the expression ensures your query operates efficiently and reduces the amount of time required to debug issues. While Filters are just a part of the overall Query Pipeline, they are integral part of the query itself. Hopefully this gives you an introduction to Query Pipeline Filters and what is possible.

If you have any questions or would like to know more, send us a message to [email protected]