Yeah, I found that I wasn’t using the right tool since telnet doesn’t allow secure connections. They day after I did this thread, I used OpenSSL and I was able to connect, so that part was lack of knowledge from my part.
I did some progress since the moment I did this thread. Doing a lot of research, I found that making a wildcard domain and adding a Wildcard SSL Certificate was recommended, so I did it and added that SSL certificate to SMTP Receiving (465 port) and IMAP services. Also, I added a DKIM key to my DNS and I believe that improved my domain security, so my mails are received by Gmail and Hotmail with no issues rn.
I also were able to login from Thunderbird just by changing the authentication method to simple (or something like that, I honestly don’t remember which one).
I’m now facing issues when trying to send mails from my MyBB installation through my SMTP server. The mails are sent and received by SMTP server, but gets rejected and logs with this message:
SSL_accept error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Talking to some MyBB devs, they said that the system would try to send the mail starting from TLS_1.3, to TLS_1.2, and so on until it reaches SSLv3.
If I do a openssl_sclient -connect 127.0.0.1:465, the server receives the connection using the TLS_1.3 protocol, as seen here:
Post-Handshake New Session Ticket arrived:
Protocol : TLSv1.3
Cipher : TLS_AES_256_GCM_SHA384
Resumption PSK: <redacted>
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
220 mail.retrolifegames.com Axigen ESMTP ready
So, I believe that it is actually receiving SMTP requests, and responding with TLS_1.3. I’m clueless why server it’s rejecting the mail commands from my MyBB installation (and seems to try with SSLv3). Hopefully you can give me a hand with this, also I know this is maybe not entirely an Axigen issue, sorry about that.