It all really depends on how did you write your Perl script (which framework, which style) and how do you invoke them (CGI, FastCGI, mod_perl, PSGI, HTTP reverse proxy, etc.) but usually every Apache handler that you add per file extension you can also add based on a directory or location.
For good examples on different ways of deploying Perl scripts on Apache using the Catalyst Web framework see: http://wiki.catalystframework.org/wiki/deployment#Apache
For examples on how to deploy Perl scripts using the Mojo framework see:
If you use CGI then your scripts will be run by
perl as long as they are executable and have something like
#!/usr/local/bin/perl in the hashbang line but it's not really recommended to use CGI any more, unless you understand the performance issues.
If you use ModPerl::Registry you can use a config like this example:
Alias /perl/ /home/httpd/perl/
and have everything in
/perl be interpreted as Perl script and run by mod_perl.
If you use PSGI or directly HTTP with a reverse proxy, you have to configure Apache to proxy everything from some location to your application using HTTP.
There are really too many ways to do it to enumerate every one of them not knowing your configuration.