So many great games were released on the PSP and most of them were RPG's; more specifically JRPG's. Some were new titles like Brave Story, but even more were glorious remakes. Persona 3 Portable, Lunar:SSH, Popolocrois (this one is probably in my top 10 PSP games). Seriously, you could play the PSP for YEARS and still not experience all of the great RPG's put on it.
Recently I pirated a copy of Popolocrois, a SRPG that was first released on the PS1 and only in Japan. It's actually two games in one but the second one seems to have been reworked. The battle system is perfect for newcomers to the JRPG genre. It plays itself pretty much during battles but for the pros out there you can turn off auto battle and have a grid to maneuver around and take out the huge bestiary of baddies.
For the "hardcore" gamers out there, Persona 3 Portable was put on the PSP. Many changes were made from the PS2 version like visual novel like scenes and less freedom to explore. To make up for that they added a female path which for people first playing this game shouldn't pick. Choose the male to experience the game normally and choose the female for your second playthrough. Also Persone 3 FES had an epilogue that I recommend watching on YouTube because P3P doesn't have that. It's not required to understand the story but it is a nice addition.
Those are really the two RPG's you should start with on the PSP. Neither are too long and don't get boring. That isn't all you could choose from though. The PSP library even includes PS1 titles if you access the PSN store. I recommend Legend of Dragoon or Xenogears for some really good older JRPG's. Have fun with you PSP and game on!