I had precisely the same problem as Will. After getting help from Mitch and doing some research I learned that (at least some) Yaesu radios require 2 stop bits for serial communication.
I had changed my baud rate on the FTDX-5000MP to 38,400 and had turned off RTS. Since my radio is at the remote site, I modified the below instructions from Mitch to 38,400 baud, 2 stop bits. Using Ham Radio Deluxe at 38,400 and RTS and DTS unchecked, it connected instantly!
Below are the instructions from Mitch. Remember to make sure all your baud rate settings are consistent, and select 2 stop bits.
CAT CHECKLIST
---------------------
IMPORTANT: In both scenarios, you MUST first install a standard serial cable between the TS-480 body and COM2 of the radio RRC!
USING A STANDARD SERIAL CABLE
============================
1) Change the CAT rate on the TS-480 to 4800 baud (menu 56). This rate keeps traffic over the internet to a minimum and faster is not necessary for CAT.
2) Change the COM2 serial settings of the radio-RRC to mode-4, 4800 baud, data bits 8, stop bits 1 and submit/apply changes.
3) Change the COM2 serial settings of the control-RRC to mode-4, 4800 baud, data bits 8, stop bits 1 and submit/apply changes.
4) Plug a serial cable in between your PC and the control-RRC COM2
5) Set your software to use the correct COM port on your PC at 4800-8-1-N
USING A USB CABLE
=================
1) Change the CAT rate on the TS-480 to 4800 baud (menu 56). This rate keeps traffic over the internet to a minimum and faster is not necessary for CAT.
2) Change the COM2 serial settings of the radio-RRC to mode-4, 4800 baud, data bits 8, stop bits 1 and submit/apply changes.
3) Install the Microbit Setup Manager if you already haven't. You don't need the problem for this, but this installs the correct drivers on your PC.
4) Change the COM2 serial settings of the control-RRC to mode-4, 4800 baud, data bits 8, stop bits 1, "use USB Com Port as COM2" to "Yes" and submit/apply changes.
5) Plug a USB cable between your PC and control-RRC. Make sure you do NOT do his before step 3 above!
6) Find out the correct port in your Windows Device Manager to use by looking for the "Microbit RRC Virtual COM Port (COM2)"
7) Set your software to use the this COM port on your PC at 4800-8-1-N.
Thanks again, Mitch!