Some how I got my CloudForms SystemEngine into a weird state. I unregistered a consumer while simultaneously removing it from the Katello UI. This caused a race condition because the UI got a 410 GONE error from Candlepin and it didn’t remove it from its database. Clearly a bug but still that left me in a state where the Systems page would show the 410 ERROR and leave me with a spinning cursor.
As this was simply a test setup I could’ve just reinstalled. But I wanted to know how this works. As you know Katello is made of many parts: Pulp, Candlepin, and Foreman.
My particular installation didn’t have Foreman, so that left: Pulp, Candlepin, Katello proper, and Elasticsearch. Since we were getting a 410 from Candlepin that means the consumer was already gone from that database
cp_consumer
table.
First, I removed the system from Katello’s database.
# psql -U katellouser katelloschema katelloschema=> delete from systems where id = 1; katelloschema=> commit; katelloschema=> \q
Then I removed the consumer from Pulp:
# mongo MongoDB shell version: 1.8.2 connecting to: test > show dbs admin (empty) local (empty) pulp_database 0.203125GB > use pulp_database switched to db pulp_database > show collections ... consumers ... > db.consumers.remove() >
What in the world did you just do? 
We can see that pulp uses the
pulp_database
. And that one of the collections was
consumers
. Running
db.consumers.help()
revealed there was a remove method. In my case I only had one system so I just did
db.consumers.remove()
. But if you have more you’ll want to specify the key:
db.consumers.remove({"_id":"62d61426-3cf9-4afb-bc3b-e0f696950adc"})
. Ok we’re done. Let’s go checkout Katello again.
UGH Still shows spinning cursor. So what else could be pointing to the fact that we had a system? Elasticsearch which is the search engine for Katello. That just needs reindexing.
# cd /usr/share/katello # rake reindex Search Indices cleared. Re-indexing ActivationKey Re-indexing Changeset Re-indexing Filter Re-indexing GpgKey Re-indexing Notice Re-indexing Organization Re-indexing Provider Re-indexing Role Re-indexing SyncPlan Re-indexing System Re-indexing TaskStatus Re-indexing User Re-indexing Repositories #
Ok let’s try one more time. Visit your systems page in Katello and now you should see that there is no longer a spinning cursor, but a blank list of systems. This is what I expected. YAY!
