Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The next UH Message Broker upgrade will bring the following changes:

ItemPreviouslyChanged to…Comments
Software version

RabbitMQ 3.7.12

Erlang 21.2.6

RabbitMQ 3.11.13

Erlang 25.3

Going forward, RabbitMQ and Erlang will be updated to the most recent versions as part of our monthly patching.
Test server

Host: esb-test2.its.hawaii.edu

Host: esb-

test-future.its.hawaii.edu (use this to test now, but complete your tests before 6/14/2023)Host: esb-

test1

.its.hawaii.edu (after the upgrade, point your test application to this)esb-test-future

.its.hawaii.edu

has data copied from production esb.hawaii.edu  since 4/18/2023 11 AM.

esb-test1.its.hawaii.edu has data copied from esb-test2.its.hawaii.edu during January 2023.

Production serverHost: esb.hawaii.edu

Host name will remain the same, but the IP address will change.  Firewall rules have been copied, and you should verify that your production application will be able to reach this new IP address at port 5671.  See 'how to test the new broker' below.


SSL Certificate2048-bit cert

4096-bit cert signed by a new intermediate CA

Subject Alternative Name (SAN) extension to support host name associated with our perceived IP address.



TLS

Versions 1.1 and 1.2 only.

No peer verification if your TLS client sends an optional client cert.

Secure renegotiation allowed.

Versions 1.2 and 1.3 only.

Peer verification performed if your TLS client sends an optional client cert.

Secure renegotiation disabled.


RabbitMQ Java ClientJava client 3.6.6 or higherAlthough we expect older clients to work, we recommend that you upgrade to the latest client

For proper security, you should eventually add code to verify our server cert/trust chain and hostname.  The Java client does not do this out of the box. See TLS and RabbitMQ Java Client

If you are already doing this, please note that there is a new CA signing our server cert, although the root CA is the same so this should not be a problem. If you run into issues, you might need to install the intermediate(s)/root cert bundle as explained in TLS and RabbitMQ Java Client

RabbitMQ Perl ClientAnyEvent::RabbitMQ v1.16

AnyEvent::RabbitMQ v1.16 or latest version

Must also patch Net::AMQP::Common and add this line after line 239:

l => \&unpack_long_long_integer



Queues

Classic queues which are mirrored and synchronized across all 3 nodes unless the queue name begins with an underscore.

All queues will be converted to quorum queues.

You do not need to change anything in your application.

More on quorum queues:  https://www.rabbitmq.com/quorum-queues.html

Timeline

  • Please test the new broker by June 14, 2023.
  • The upgrade will happen on July 9, 2023

How to test the new broker

  1. This first step is optional, but it will save you from a lot of troubleshooting if you are able to perform itVerify that you won't have issues with the new firewall:
    1. Login to the system where your production application runs and connects to the UH Message Broker:
    2. Run this openssl command:
      • openssl s_client -connect 128.171.138.176:5671
        The first few lines should look like this

        No Format
        CONNECTED(00000003)
        depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
        verify return:1
        depth=1 C = US, O = Internet2, CN = InCommon RSA Server CA 2
        verify return:1
        depth=0 C = US, ST
        = Hawaii, O = University of Hawaii at Manoa, CN = esb-test-future.its.hawaii.edu
        verify return:1
        write W BLOCK
        ---
        Certificate chain
         0 s:/C=US/ST=Hawaii/O=University of Hawaii at Manoa/CN=esb-test-future.its.hawaii.edu
           i:/C=US/O=Internet2/CN=InCommon RSA Server CA 2
         1 s:/C=US/O=Internet2/CN=InCommon RSA Server CA 2
           i:/C=US/ST=New Jersey/L=Jersey
        City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
         2 s:/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
           i:/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services
         3 s:/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services
           i:/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services
        ---
        
        
    Set up

  2. Set up or go to a test instance of your applicationPreferably set up this test instance on the same host that runs your production application (this will confirm that our firewall will allow you to connect after we upgrade).
    • If you need to test from a different hosthost that has never connected to esb-test1 or esb-test2, please send us the IP address of this test host to its-iam-help@lists.hawaii.edu and wait for us to allow it through our firewall. You can also run the openssl command from step 1 above to check whether you are already allowed to connect.
    • (warning) Regardless of where it runs, this test instance must not make production data changes.  Be sure to keep production and test instances of your application completely separate and independent.
  3. Connect the test instance of your application to the following test broker:
    • Test broker host: esb-test-futuretest1.its.hawaii.edu
    • Test broker port: 5671 (TLS 1.1 no longer supported, must use TLS 1.2 or 1.3)
    • Test broker account and password: (same as production brokeresb-test2, contact us if you've never connected to esb-test2 before)
    • Test broker vhost: (add future- prefix to whatever vhost you are using in production, e.g. if vhost is uhims, then use future-uhims in this test brokersame as esb-test2)
    • Test broker exchanges and queues: (same names as in production broker)If your application production environment has been using TLS 1.1, you must now use TLS 1.2 or 1.3.as esb-test2)
  4. Verify that the test instance of your application can consume messages from the above test broker.
    • You can skip this test if your application only publishes to the broker (only a couple of applications do this)
    • For these tests, remember to change the vhost to have the future- prefix.  The queue name should be the same as what you use in production
    • Your queues are getting copies of messages sent to the production broker since 4/18/2023 11:00 AM.
      • (warning) You should test consuming messages from these queues because copied messages have additional metadata (message headers).  During the cut over to the new broker, we will need to copy any unconsumed messages from the old queues to the new queues.  Therefore, you should verify that you can consume these copied messages and that their extra headers are not a problem for you.
    If your application publishes to the broker, and we have not previously contacted you to test against this new version of the broker, please go ahead and test publishing.
  5. Remember to add the future- prefix to the vhost name for these tests.
  6. If you do not have access to a queue to confirm that your messages are being published, please contact us at the email address below.

Have more questions? Contact its-iam-help@lists.hawaii.edu

...