JarCastingSolr OSGi
This project goal is to be able to run Apache Solr in OSGi environment and bring the OSGi dynamic lifecycle to the Solr world.
|
Note
|
There is no all of the Solr features supported yet, clustering and sharding haven’t been tested. In theory every piece of feature can be implemented in the future. Cellar based SolCloud implementation is planned. Now only configSet based cores can be used. |
How it works?
There are a declarative component called SolrCoreContainerManager which monitors the following items:
-
solr.core or solr.core-<pid> configuration PID-s in ConfigurationAdmin service. This configuration defines a core cofigurations. The properties are very same as can be used in CoreAdmin API. Custom properties can be defined in admin-requests, 'property' prefix can be used on custom properties. Two mandatory properties are: name and configSet.
-
All bundles which have the Solr-Configset tag in META-INF/MANIFEST.MF. It loades configSets to CoreContainer. The cores can use thats configSets as a template. The configSets are stored to an in-Memory filesystem which is used to load files by solr.
The manager creates SolrClient instances when the configSet loaded which referenced by solr.core confugurations. In the future the core definitions which does not have configSet can be created as normal cores, but for that some file based resource management implementation is required.
When a core config or configSet bundle removed the related solrClients are stopping. The data directory is not removed, so it can be reused later.
Installation and usage
Any OSGi 6.0 container canbe used for deployment. It is tested with Apache Karaf 4.1.2.
To install in karaf console:
feature:repo-add cxf feature:repo-add mvn:hu.blackbelt/solr-osgi-feature/<version>/xml/features feature:install -v solr-http
To be able to use the SolrCoreContainerManager’s functionality have to create a configuration with pid solr.corecontainer
Create a file solr.corecontainer.cfg in karaf/deploy with the following content:
solrHome=/tmp/solr
karaf@root()> log:tail 2018-08-22 00:33:01,207 | INFO | tures-1-thread-1 | HttpServiceFactoryImpl | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Binding bundle: [org.restlet.ext.servlet [185]] to http service 2018-08-22 00:33:01,208 | DEBUG | tures-1-thread-1 | HttpServiceStarted | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | Creating http service for: org.restlet.ext.servlet [185] 2018-08-22 00:33:01,210 | DEBUG | tures-1-thread-1 | HttpServiceProxy | 181 - org.ops4j.pax.web.pax-web-runtime - 6.0.6 | HttpServiceProxy created for HttpService org.ops4j.pax.web.service.internal.HttpServiceStarted@6de4a397 for bundle org.restlet.ext.servlet [185] 2018-08-22 00:33:01,214 | INFO | tures-1-thread-1 | FeaturesServiceImpl | 9 - org.apache.karaf.features.core - 4.1.2 | hu.blackbelt.solr-osgi-services/1.0.0.SNAPSHOT 2018-08-22 00:33:01,342 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | ___ _ Welcome to Apache Solr? version 7.0.1 2018-08-22 00:33:01,344 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | / __| ___| |_ _ Starting in standalone mode 2018-08-22 00:33:01,344 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | \__ \/ _ \ | '_| 2018-08-22 00:33:01,358 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | |___/\___/_|_| Start time: 2018-08-21T22:33:01.345Z 2018-08-22 00:33:01,459 | INFO | tures-1-thread-1 | SolrResourceLoader | 147 - org.apache.solr - 7.0.1.1 | solr home defaulted to 'solr/' (could not find system property or JNDI) 2018-08-22 00:33:01,867 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:01,869 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:01,882 | INFO | tures-1-thread-1 | SolrXmlConfig | 147 - org.apache.solr - 7.0.1.1 | MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44, but no JMX reporters were configured - adding default JMX reporter. 2018-08-22 00:33:02,120 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,120 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,121 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Get instance path: / 2018-08-22 00:33:02,382 | WARN | tures-1-thread-1 | LogWatcher | 147 - org.apache.solr - 7.0.1.1 | Unable to read SLF4J version. LogWatcher will be disabled: java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder 2018-08-22 00:33:02,390 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ZookeeperInfoHandler Expected tyoe: org.apache.solr.handler.admin.ZookeeperInfoHandler 2018-08-22 00:33:02,402 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CollectionsHandler Expected tyoe: org.apache.solr.handler.admin.CollectionsHandler 2018-08-22 00:33:04,662 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.InfoHandler Expected tyoe: org.apache.solr.handler.admin.InfoHandler 2018-08-22 00:33:04,672 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.CoreAdminHandler Expected tyoe: org.apache.solr.handler.admin.CoreAdminHandler 2018-08-22 00:33:04,690 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.ConfigSetsHandler Expected tyoe: org.apache.solr.handler.admin.ConfigSetsHandler 2018-08-22 00:33:04,696 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsHandler Expected tyoe: org.apache.solr.handler.admin.MetricsHandler 2018-08-22 00:33:04,697 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.handler.admin.MetricsCollectorHandler Expected tyoe: org.apache.solr.handler.admin.MetricsCollectorHandler 2018-08-22 00:33:04,706 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.cloud.autoscaling.AutoScalingHandler Expected tyoe: org.apache.solr.cloud.autoscaling.AutoScalingHandler 2018-08-22 00:33:04,718 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,751 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,752 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,753 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,753 | INFO | tures-1-thread-1 | OsgiSolrResourceLoader | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | NewInstance: org.apache.solr.metrics.reporters.SolrJmxReporter Expected tyoe: org.apache.solr.metrics.SolrMetricReporter 2018-08-22 00:33:04,754 | INFO | tures-1-thread-1 | SolrJmxReporter | 147 - org.apache.solr - 7.0.1.1 | JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67117f44 2018-08-22 00:33:04,793 | INFO | tures-1-thread-1 | CorePropertiesLocator | 147 - org.apache.solr - 7.0.1.1 | Found 0 core definitions underneath /
To add a configSet template, copy the solr-osgi-example-configsets-exampleCollection projects jar to karaf/deploy
2018-08-22 00:33:04,850 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Solr content found in bundle hu.blackbelt.solr-osgi-example-configsets-exampleCollection Registering 2018-08-22 00:33:04,852 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/lang/stopwords_en.txt to configsets/exampleCollection/conf/lang/stopwords_en.txt 2018-08-22 00:33:04,858 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/params.json to configsets/exampleCollection/conf/params.json 2018-08-22 00:33:04,859 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/protwords.txt to configsets/exampleCollection/conf/protwords.txt 2018-08-22 00:33:04,859 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/schema.xml to configsets/exampleCollection/conf/schema.xml 2018-08-22 00:33:04,860 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/solrconfig.xml to configsets/exampleCollection/conf/solrconfig.xml 2018-08-22 00:33:04,861 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/stopwords.txt to configsets/exampleCollection/conf/stopwords.txt 2018-08-22 00:33:04,861 | INFO | tures-1-thread-1 | SolrCoreContainerManager | 76 - hu.blackbelt.solr-osgi-services - 1.0.0.SNAPSHOT | Copy file bundle://52.0:0/configsets/exampleCollection/conf/synonyms.txt to configsets/exampleCollection/conf/synonyms.txt 2018-08-22 00:33:04,869 | INFO | tures-1-thread-1 | SolrCoreContainerMan
The configSet files processed and copied to in-memory store.
To create a core based on a configSet, create a configuration with PID solr.core-<pid>
In karaf create a file solr.core-test1.cfg in karaf/deploy with the following content:
name=test1 configSet=exampleCollection
HTTP access
To install HTTP service create configuration with PID solr.http
In karaf create a file solr.http.cfg in karaf/deploy with the following content:
contextRoot=/solr
After the Solr Admin can be accessed http://localhost:8181/solr/
Core container parameters
Core containers have very same parameters as the original ones, but not all of them can be used now.
