We created a jounal entry workflow. However, those that are not supposed to be approvers, were able to approve the journals through the Approve Journals page

Check what you're setting as the next approver value. I suspect if it's not set then that can open up the JE to be approved by anyone with the permission

Figured it out. Thank you! It was because I wasn't able to setup the Accounting Pref > Approval Routing on JE. Thank you!