Solr, Zookeeper & Drupal 10 Integration Documentation

This document details the setup and configuration of a Solr search server (in SolrCloud mode) used for Drupal 10’s Search API integration. It covers installation prerequisites, Solr and ZooKeeper configuration, Apache reverse proxy setup with Basic Authentication, and Drupal-specific settings.

When securing access to Solr, there are two primary approaches. One option is to use a reverse proxy (such as Apache or Nginx) to enforce HTTP Basic Authentication, which provides a robust barrier against unauthorized access without requiring modifications to Solr itself. In this scenario, the native Solr security configuration (via a security.json file) may not be necessary, as the proxy handles all authentication and access control. Alternatively, you can configure Solr's built-in security directly through the security.json file, ensuring that every access attempt is authenticated—even if Solr is reached directly. The choice depends on your network architecture and security needs: if Solr is only accessible through a properly secured proxy, you might opt for the simpler proxy-based method; if additional layers of security are desired, especially in environments where direct access is possible, enabling Solr's native security is recommended.

It's possible to launch both Solr and ZooKeeper as the "solr" user, especially in a testing or proof-of-concept environment. However, in production environments it’s generally best practice to run ZooKeeper as a separate system service (often under its own dedicated user) and then start Solr (as the "solr" user) configured to connect to that ZooKeeper service. Running them separately improves resource isolation, security, and manageability—ensuring that ZooKeeper remains available independently of Solr’s lifecycle.

1. Overview

Purpose:
Provide search indexing for Drupal 10 content using Apache Solr in Cloud mode.

Components:

Apache Solr (v8.11.4)
Apache ZooKeeper (v3.8.4)
Java (installed system-wide)
Drupal 10 with Search API module (contributed)
Apache HTTP Server (reverse proxy with Basic Auth)

Key Challenges Addressed:

Running Solr in Cloud mode
Configuring Apache reverse proxy to expose the admin panel
Securing Solr with Basic Authentication
Loading and applying custom Drupal search configurations to Solr

2. Prerequisites

Operating System:
A Linux distribution with appropriate package management.

Installed Software:

Java (JDK/JRE)
Apache Solr (installed under /opt/solr-8.11.4/)
Apache ZooKeeper (installed under /opt/apache-zookeeper-3.8.4-bin/)

> sudo useradd -r -s /bin/false zookeeper
> sudo nano /etc/systemd/system/zookeeper.service

[Unit]
Description=Apache ZooKeeper Server
Documentation=http://zookeeper.apache.org
After=network.target

[Service]
Type=simple
User=zookeeper
Group=zookeeper
ExecStart=/opt/zookeeper/bin/zkServer.sh start-foreground
ExecStop=/opt/zookeeper/bin/zkServer.sh stop
Restart=on-failure
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target

> sudo -u solr /opt/zookeeper/bin/zkServer.sh start
> sudo -u solr /opt/zookeeper/bin/zkServer.sh status
> sudo -u solr /opt/zookeeper/bin/zkCli.sh -server 127.0.0.1:2181

sudo systemctl daemon-reload
sudo systemctl enable zookeeper
sudo systemctl start zookeeper
sudo systemctl status zookeeper

sudo mkdir -p /var/lib/zookeeper/data
sudo chown -R zookeeper:zookeeper /var/lib/zookeeper
sudo -u solr bin/solr create -c drupal_search -n drupal_search

User Setup:

Dedicated users (e.g., solr/zookeeper) with the required permissions.

Directory Structure Overview:
/opt/
├── solr -> /opt/solr-8.11.4/
└── zookeeper -> /opt/apache-zookeeper-3.8.4-bin

3. Solr Cloud Mode Configuration

3.1. Uploading Custom Configurations to ZooKeeper

Remove existing configuration (if needed):

Upload custom Drupal Solr configuration:

Verify available configurations:

(Optional) Upload configuration for _default if needed:

3.2. Managing the Solr Cloud Instance

Check Solr status:

Restart Solr in Cloud mode:

4. Apache Reverse Proxy Configuration

4.1. Apache Virtual Host Setup

Example Virtual Host configuration snippet:

apacheconf

<VirtualHost *:80>
ServerName solr.example.com
ProxyPreserveHost On
ProxyPass /solr http://localhost:8983/solr
ProxyPassReverse /solr http://localhost:8983/solr
# Optional: Force HTTPS if required
# RewriteEngine On
# RewriteCond %{HTTPS} off
# RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
<Location /solr>
AuthType Basic
AuthName "Solr Admin Access"
AuthUserFile /etc/apache2/.htpasswd_solr
Require valid-user
</Location> </VirtualHost>

4.2. Securing with Basic Authentication

Steps:

Create or update the .htpasswd_solr file (using Apache’s htpasswd tool).
Verify permissions so Apache can read the file.
Reload/restart Apache to apply the changes.

5. Drupal Configuration

5.1. Search API Module Settings

Endpoint Configuration:
In Drupal’s Search API configuration, set the Solr endpoint URL to match the reverse proxy (e.g., http://solr.example.com/solr).

Authentication:
If your reverse proxy enforces Basic Auth, include the credentials in the Drupal connector settings.

5.2. Custom Schema & Configuration

Custom Drupal Configurations:
Ensure that your custom configuration (uploaded via ZooKeeper) is compatible with Drupal’s search schema.

Troubleshooting Connection Issues:

Verify that the correct configuration name (e.g., drupal_search) is referenced.
Check network connectivity from the Drupal host to the Solr endpoint.
Consult Solr and Apache logs for authentication or proxy errors.

6. Troubleshooting & Verification

6.1. Solr Verification

Commands to check Solr Cloud:

Review Solr Logs:
Located typically in the /opt/solr-8.11.4/server/logs directory.

6.2. Apache Reverse Proxy Logs

Location:
Usually found in /var/log/apache2/ (or the location defined in your Apache configuration).

6.3. Drupal Connectivity

Drupal Log Messages:
Check Recent log messages under Reports > Recent log messages in the Drupal admin interface.

Module Debugging:
Enable error logging for the Search API module if needed.

7. Additional Configuration Files & Details

Solr Configuration Files:

solr.xml
schema.xml or managed-schema (depending on your setup)
Config files under /tmp/drupal_solr_config/ (the directory you uploaded)

ZooKeeper Configuration Files:

zoo.cfg
Environment settings for the solr user

Apache Configuration Files:

Virtual Host files (e.g., /etc/apache2/sites-available/solr.conf)
Authentication files (e.g., .htpasswd_solr)

8. Summary & Best Practices

Deployment Best Practices:

Test configuration changes in a staging environment before production.
Regularly back up configuration files and ZooKeeper data.
Monitor Solr, ZooKeeper, and Apache logs for early error detection.

Documentation Updates:
Keep this document updated with any changes to the configurations or versions used.

References & Further Reading:

9. Appendix

Command Summary:
A quick reference for the commands used:

# Remove previous config from ZooKeeper
sudo -u solr bin/solr zk rm -r /configs/drupal_search -z localhost:2181
# create a new collection
sudo -u solr bin/solr create -c drupal_search -n drupal_search
# Upload new custom configuration
sudo -u solr bin/solr zk upconfig -z localhost:2181 -d /tmp/drupal_solr_config -n drupal_search
# Verify configs
sudo -u solr bin/solr zk ls /configs -z localhost:2181
# Optionally update default config

 sudo -u solr bin/solr zk upconfig -z localhost:2181 -d /tmp/drupal_solr_config -n _default # Check Solr status sudo -u solr bin/solr status # Restart Solr in cloud mode sudo -u solr bin/solr restart -c

Directory Structure Reminder: