なんかperlでライブラリを提供されたので、perlのスクリプトを書くことになってしまい、
cpanとかよくわからないけど触ってみたのでブログ書いてみた。
用意してもらったライブラリはHadoopにwebHDFSでファイルをアップするためのラッパー。
# perl test.pl Can't locate Net/Hadoop/WebHDFS.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/share/perl5/sample/WebHDFS.pm line 6. BEGIN failed--compilation aborted at /usr/local/share/perl5/sample/WebHDFS.pm line 6. Compilation failed in require at test.pl line 1. BEGIN failed--compilation aborted at test.pl line 1.とりあえず使ってみたら、Can't locate Net/Hadoop/WebHDFS.pmと言われた。
モジュールがないんだろうなということで、cpanで入れてみた。
途中「yes」と入れるところがあったりするが、ずらずらーっと一気に流れていく。
# cpan Net::Hadoop::WebHDFS CPAN is the world-wide archive of perl resources. It consists of about 300 sites that all replicate the same contents around the globe. Many countries have at least one CPAN site already. The resources found on CPAN are easily accessible with the CPAN.pm module. If you want to use CPAN.pm, lots of things have to be configured. Fortunately, most of them can be determined automatically. If you prefer the automatic configuration, answer 'yes' below. If you prefer to enter a dialog instead, you can answer 'no' to this question and I'll let you configure in small steps one thing after the other. (Note: you can revisit this dialog anytime later by typing 'o conf init' at the cpan prompt.) Would you like me to configure as much as possible automatically? [yes] The following questions are intended to help you with the configuration. The CPAN module needs a directory of its own to cache important index files and maybe keep a temporary mirror of CPAN files. This may be a site-wide or a personal directory. First of all, I'd like to create this directory. Where?CPAN build and cache directory? [/root/.cpan] Unless you are accessing the CPAN on your filesystem via a file: URL, CPAN.pm needs to keep the source files it downloads somewhere. Please supply a directory where the downloaded files are to be kept. Download target directory? [/root/.cpan/sources] Directory where the build process takes place? [/root/.cpan/build] Until version 1.88 CPAN.pm never trusted the contents of the build_dir directory between sessions. Since 1.88_58 CPAN.pm has a YAML-based mechanism that makes it possible to share the contents of the build_dir/ directory between different sessions with the same version of perl. People who prefer to test things several days before installing will like this feature because it safes a lot of time. If you say yes to the following question, CPAN will try to store enough information about the build process so that it can pick up in future sessions at the same state of affairs as it left a previous session. Store and re-use state information about distributions between CPAN.pm sessions? [no] CPAN.pm can store customized build environments based on regular expressions for distribution names. These are YAML files where the default options for CPAN.pm and the environment can be overridden and dialog sequences can be stored that can later be executed by an Expect.pm object. The CPAN.pm distribution comes with some prefab YAML files that cover sample distributions that can be used as blueprints to store one own prefs. Please check out the distroprefs/ directory of the CPAN.pm distribution to get a quick start into the prefs system. Directory where to store default options/environment/dialogs for building modules that need some customization? [/root/.cpan/prefs] Normally CPAN.pm keeps config variables in memory and changes need to be saved in a separate 'o conf commit' command to make them permanent between sessions. If you set the 'auto_commit' option to true, changes to a config variable are always automatically committed to disk. Always commit changes to config variables to disk? [no] CPAN.pm can limit the size of the disk area for keeping the build directories with all the intermediate files. Cache size for build directory (in MB)? [100] The CPAN indexes are usually rebuilt once or twice per hour, but the typical CPAN mirror mirrors only once or twice per day. Depending on the quality of your mirror and your desire to be on the bleeding edge, you may want to set the following value to more or less than one day (which is the default). It determines after how many days CPAN.pm downloads new indexes. Let the index expire after how many days? [1] By default, each time the CPAN module is started, cache scanning is performed to keep the cache size in sync. To prevent this, answer 'never'. Perform cache scanning (atstart or never)? [atstart] To considerably speed up the initial CPAN shell startup, it is possible to use Storable to create a cache of metadata. If Storable is not available, the normal index mechanism will be used. Note: this mechanism is not used when use_sqlite is on and SQLLite is running. Cache metadata (yes/no)? [yes] CPAN::SQLite is a layer between the index files that are downloaded from the CPAN and CPAN.pm that speeds up metadata queries and reduces memory consumption of CPAN.pm considerably. Use CPAN::SQLite if available? (yes/no)? [no] The CPAN module can detect when a module which you are trying to build depends on prerequisites. If this happens, it can build the prerequisites for you automatically ('follow'), ask you for confirmation ('ask'), or just ignore them ('ignore'). Please set your policy to one of the three values. Policy on building prerequisites (follow, ask or ignore)? [ask] When a module declares another one as a 'build_requires' prerequisite this means that the other module is only needed for building or testing the module but need not be installed permanently. In this case you may wish to install that other module nonetheless or just keep it in the 'build_dir' directory to have it available only temporarily. Installing saves time on future installations but makes the perl installation bigger. You can choose if you want to always install (yes), never install (no) or be always asked. In the latter case you can set the default answer for the question to yes (ask/yes) or no (ask/no). Policy on installing 'build_requires' modules (yes, no, ask/yes, ask/no)? [ask/yes] CPAN packages can be digitally signed by authors and thus verified with the security provided by strong cryptography. The exact mechanism is defined in the Module::Signature module. While this is generally considered a good thing, it is not always convenient to the end user to install modules that are signed incorrectly or where the key of the author is not available or where some prerequisite for Module::Signature has a bug and so on. With the check_sigs parameter you can turn signature checking on and off. The default is off for now because the whole tool chain for the functionality is not yet considered mature by some. The author of CPAN.pm would recommend setting it to true most of the time and turning it off only if it turns out to be annoying. Note that if you do not have Module::Signature installed, no signature checks will be performed at all. Always try to check and verify signatures if a SIGNATURE file is in the package and Module::Signature is installed (yes/no)? [no] The goal of the CPAN Testers project (http://testers.cpan.org/) is to test as many CPAN packages as possible on as many platforms as possible. This provides valuable feedback to module authors and potential users to identify bugs or platform compatibility issues and improves the overall quality and value of CPAN. One way you can contribute is to send test results for each module that you install. If you install the CPAN::Reporter module, you have the option to automatically generate and email test reports to CPAN Testers whenever you run tests on a CPAN package. See the CPAN::Reporter documentation for additional details and configuration settings. If your firewall blocks outgoing email, you will need to configure CPAN::Reporter before sending reports. Email test reports if CPAN::Reporter is installed (yes/no)? [no] When a distribution has already been tested by CPAN::Reporter on this machine, CPAN can skip the test phase and just rely on the test report history instead. Note that this will not apply to distributions that failed tests because of missing dependencies. Also, tests can be run regardless of the history using "force". Do you want to rely on the test report history (yes/no)? [no] At the time of this writing (2009-03) there are three YAML implementations working: YAML, YAML::Syck, and YAML::XS. The latter two are faster but need a C compiler installed on your system. There may be more alternative YAML conforming modules. When I tried two other players, YAML::Tiny and YAML::Perl, they seemed not powerful enough to work with CPAN.pm. This may have changed in the meantime. Which YAML implementation would you prefer? [YAML] Warning (maybe harmless): 'YAML' not installed. Both YAML.pm and YAML::Syck are capable of deserialising code. As this requires a string eval, which might be a security risk, you can use this option to enable or disable the deserialisation of code via CPAN::DeferredCode. (Note: This does not work under perl 5.6) Do you want to enable code deserialisation (yes/no)? [no] The CPAN module will need a few external programs to work properly. Please correct me, if I guess the wrong path for a program. Don't panic if you do not have some of them, just press ENTER for those. To disable the use of a program, you can type a space followed by ENTER. Where is your bzip2 program? [/usr/bin/bzip2] Where is your gzip program? [/bin/gzip] Where is your tar program? [/bin/tar] Where is your unzip program? [/usr/bin/unzip] Where is your make program? [/usr/bin/make] Where is your curl program? [/usr/bin/curl] Warning: lynx not found in PATH[/usr/local/maven/bin;/usr/local/sbin;/usr/local/bin;/sbin;/bin;/usr/sbin;/usr/bin;/home/user/core/bin;/usr/local/groovy/bin;/usr/java/default/bin;/root/bin;/opt/infra/ruby/bin] Where is your lynx program? [] Where is your wget program? [/usr/bin/wget] Warning: ncftpget not found in PATH[/usr/local/maven/bin;/usr/local/sbin;/usr/local/bin;/sbin;/bin;/usr/sbin;/usr/bin;/home/user/core/bin;/usr/local/groovy/bin;/usr/java/default/bin;/root/bin;/opt/infra/ruby/bin] Where is your ncftpget program? [] Warning: ncftp not found in PATH[/usr/local/maven/bin;/usr/local/sbin;/usr/local/bin;/sbin;/bin;/usr/sbin;/usr/bin;/home/user/core/bin;/usr/local/groovy/bin;/usr/java/default/bin;/root/bin;/opt/infra/ruby/bin] Where is your ncftp program? [] Warning: ftp not found in PATH[/usr/local/maven/bin;/usr/local/sbin;/usr/local/bin;/sbin;/bin;/usr/sbin;/usr/bin;/home/user/core/bin;/usr/local/groovy/bin;/usr/java/default/bin;/root/bin;/opt/infra/ruby/bin] Where is your ftp program? [] Where is your gpg program? [/usr/bin/gpg] Where is your patch program? [/usr/bin/patch] Warning: applypatch not found in PATH[/usr/local/maven/bin;/usr/local/sbin;/usr/local/bin;/sbin;/bin;/usr/sbin;/usr/bin;/home/usercore/bin;/usr/local/groovy/bin;/usr/java/default/bin;/root/bin;/opt/infra/ruby/bin] Where is your applypatch program? [] What is your favorite pager program? [/usr/bin/less] What is your favorite shell? [/bin/bash] When CPAN.pm uses the tar command, which switch for the verbosity shall be used? Choose 'none' for quiet operation, 'v' for file name listing, 'vv' for full listing. Tar command verbosity level (none or v or vv)? [v] When CPAN.pm loads a module it needs for some optional feature, it usually reports about module name and version. Choose 'v' to get this message, 'none' to suppress it. Verbosity level for loading modules (none or v)? [v] When CPAN.pm extends @INC via PERL5LIB, it prints a list of directories added (or a summary of how many directories are added). Choose 'v' to get this message, 'none' to suppress it. Verbosity level for PERL5LIB changes (none or v)? [v] When the CPAN shell is started it normally displays a greeting message that contains the running version and the status of readline support. Do you want to turn this message off? [no] When you have Module::Build installed and a module comes with both a Makefile.PL and a Build.PL, which shall have precedence? The main two standard installer modules are the old and well established ExtUtils::MakeMaker (for short: EUMM) which uses the Makefile.PL. And the next generation installer Module::Build (MB) which works with the Build.PL (and often comes with a Makefile.PL too). If a module comes only with one of the two we will use that one but if both are supplied then a decision must be made between EUMM and MB. See also http://rt.cpan.org/Ticket/Display.html?id=29235 for a discussion about the right default. Or, as a third option you can choose RAND which will make a random decision (something regular CPAN testers will enjoy). In case you can choose between running a Makefile.PL or a Build.PL, which installer would you prefer (EUMM or MB or RAND)? [MB] Every Makefile.PL is run by perl in a separate process. Likewise we run 'make' and 'make install' in separate processes. If you have any parameters (e.g. PREFIX, UNINST or the like) you want to pass to the calls, please specify them here. If you don't understand this question, just press ENTER. Typical frequently used settings: PREFIX=~/perl # non-root users (please see manual for more hints) Parameters for the 'perl Makefile.PL' command? [INSTALLDIRS=site] Parameters for the 'make' command? Typical frequently used setting: -j3 # dual processor system (on GNU make) Your choice: [] Do you want to use a different make command for 'make install'? Cautious people will probably prefer: su root -c make or sudo make or /path1/to/sudo -u admin_account /path2/to/make or some such. Your choice: [/usr/bin/make] Parameters for the 'make install' command? Typical frequently used setting: UNINST=1 # to always uninstall potentially conflicting files Your choice: [] A Build.PL is run by perl in a separate process. Likewise we run './Build' and './Build install' in separate processes. If you have any parameters you want to pass to the calls, please specify them here. Typical frequently used settings: --install_base /home/xxx # different installation directory Parameters for the 'perl Build.PL' command? [--installdirs site] Parameters for the './Build' command? Setting might be: --extra_linker_flags -L/usr/foo/lib # non-standard library location Your choice: [] Do you want to use a different command for './Build install'? Sudo users will probably prefer: su root -c ./Build or sudo ./Build or /path1/to/sudo -u admin_account ./Build or some such. Your choice: [./Build] Parameters for the './Build install' command? Typical frequently used setting: --uninst 1 # uninstall conflicting files Your choice: [] Sometimes you may wish to leave the processes run by CPAN alone without caring about them. Because the Makefile.PL or the Build.PL sometimes contains question you're expected to answer, you can set a timer that will kill a 'perl Makefile.PL' process after the specified time in seconds. If you set this value to 0, these processes will wait forever. This is the default and recommended setting. Timeout for inactivity during {Makefile,Build}.PL? [0] Normaly, CPAN.pm continues processing the full list of targets and dependencies, even if one of them fails. However, you can specify that CPAN should halt after the first failure. Do you want to halt on failure (yes/no)? [no] If you're accessing the net via proxies, you can specify them in the CPAN configuration or via environment variables. The variable in the $CPAN::Config takes precedence. Your ftp_proxy? [] Your http_proxy? [] Your no_proxy? [] Shall we always set the FTP_PASSIVE environment variable when dealing with ftp download (yes/no)? [yes] CPAN.pm changes the current working directory often and needs to determine its own current working directory. Per default it uses Cwd::cwd but if this doesn't work on your system for some reason, alternatives can be configured according to the following table: cwd Cwd::cwd getcwd Cwd::getcwd fastcwd Cwd::fastcwd backtickcwd external command cwd Preferred method for determining the current working directory? [cwd] The prompt of the cpan shell can contain the current command number for easier tracking of the session or be a plain string. Do you want the command number in the prompt (yes/no)? [yes] When using Term::ReadLine, you can turn ornaments on so that your input stands out against the output from CPAN.pm. Do you want to turn ornaments on? [yes] The next option deals with the charset (aka character set) your terminal supports. In general, CPAN is English speaking territory, so the charset does not matter much but some CPAN have names that are outside the ASCII range. If your terminal supports UTF-8, you should say no to the next question. If it expects ISO-8859-1 (also known as LATIN1) then you should say yes. If it supports neither, your answer does not matter because you will not be able to read the names of some authors anyway. If you answer no, names will be output in UTF-8. Your terminal expects ISO-8859-1 (yes/no)? [yes] If you have one of the readline packages (Term::ReadLine::Perl, Term::ReadLine::Gnu, possibly others) installed, the interactive CPAN shell will have history support. The next two questions deal with the filename of the history file and with its size. If you do not want to set this variable, please hit SPACE RETURN to the following question. If you have one of the readline packages (Term::ReadLine::Perl, Term::ReadLine::Gnu, possibly others) installed, the interactive CPAN shell will have history support. The next two questions deal with the filename of the history file and with its size. If you do not want to set this variable, please hit SPACE RETURN to the following question. File to save your history? [/root/.cpan/histfile] Number of lines to save? [100] The 'd' and the 'm' command normally only show you information they have in their in-memory database and thus will never connect to the internet. If you set the 'show_upload_date' variable to true, 'm' and 'd' will additionally show you the upload date of the module or distribution. Per default this feature is off because it may require a net connection to get at the upload date. Always try to show upload date with 'd' and 'm' command (yes/no)? [no] During the 'r' command CPAN.pm finds modules without version number. When the command finishes, it prints a report about this. If you want this report to be very verbose, say yes to the following variable. Show all individual modules that have no $VERSION? [no] During the 'r' command CPAN.pm finds modules with a version number of zero. When the command finishes, it prints a report about this. If you want this report to be very verbose, say yes to the following variable. Show all individual modules that have a $VERSION of zero? [no] If you have never defined your own C in your configuration then C will be hesitant to use the built in default sites for downloading. It will ask you once per session if a connection to the internet is OK and only if you say yes, it will try to connect. But to avoid this question, you can choose your favorite download sites once and get away with it. Or, if you have no favorite download sites answer yes to the following question. If no urllist has been chosen yet, would you prefer CPAN.pm to connect to the built-in default sites without asking? (yes/no)? [yes] Autoconfigured everything but 'urllist'. Please call 'o conf init urllist' to configure your CPAN server(s) now! commit: wrote '/usr/share/perl5/CPAN/Config.pm' CPAN: Storable loaded ok (v2.20) CPAN: LWP::UserAgent loaded ok (v5.833) Warning: no success downloading '/root/.cpan/sources/authors/01mailrc.txt.gz.tmp30757'. Giving up on it. at /usr/share/perl5/CPAN/Index.pm line 225 Fetching with LWP: http://www.perl.org/CPAN/authors/01mailrc.txt.gz Going to read '/root/.cpan/sources/authors/01mailrc.txt.gz' ............................................................................DONE Fetching with LWP: http://www.perl.org/CPAN/modules/02packages.details.txt.gz Going to read '/root/.cpan/sources/modules/02packages.details.txt.gz' Database was generated on Thu, 19 Jun 2014 05:41:02 GMT .............. New CPAN.pm version (v2.05) available. [Currently running version is v1.9402] You might want to try install CPAN reload cpan to both upgrade CPAN.pm and run the new version without leaving the current session. ..............................................................DONE Fetching with LWP: http://www.perl.org/CPAN/modules/03modlist.data.gz Going to read '/root/.cpan/sources/modules/03modlist.data.gz' DONE Going to write /root/.cpan/Metadata Running install for module 'Net::Hadoop::WebHDFS' CPAN: Data::Dumper loaded ok (v2.124) 'YAML' not installed, falling back to Data::Dumper and Storable to read prefs '/root/.cpan/prefs' Running make for T/TA/TAGOMORIS/Net-Hadoop-WebHDFS-0.5.tar.gz Fetching with LWP: http://www.perl.org/CPAN/authors/id/T/TA/TAGOMORIS/Net-Hadoop-WebHDFS-0.5.tar.gz CPAN: Digest::SHA loaded ok (v5.47) Fetching with LWP: http://www.perl.org/CPAN/authors/id/T/TA/TAGOMORIS/CHECKSUMS Checksum for /root/.cpan/sources/authors/id/T/TA/TAGOMORIS/Net-Hadoop-WebHDFS-0.5.tar.gz ok CPAN: Archive::Tar loaded ok (v1.58) Net-Hadoop-WebHDFS-0.5/Build.PL Net-Hadoop-WebHDFS-0.5/Changes Net-Hadoop-WebHDFS-0.5/LICENSE Net-Hadoop-WebHDFS-0.5/META.json Net-Hadoop-WebHDFS-0.5/META.yml Net-Hadoop-WebHDFS-0.5/README.md Net-Hadoop-WebHDFS-0.5/cpanfile Net-Hadoop-WebHDFS-0.5/lib/Net/Hadoop/WebHDFS.pm Net-Hadoop-WebHDFS-0.5/t/Net-Hadoop-WebHDFS.t Net-Hadoop-WebHDFS-0.5/MANIFEST CPAN: File::Temp loaded ok (v0.22) CPAN.pm: Going to build T/TA/TAGOMORIS/Net-Hadoop-WebHDFS-0.5.tar.gz Can't locate CPAN/Meta.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at Build.PL line 15. BEGIN failed--compilation aborted at Build.PL line 15. Warning: No success on command[/usr/bin/perl Build.PL --installdirs site] Warning (usually harmless): 'YAML' not installed, will not store persistent state TAGOMORIS/Net-Hadoop-WebHDFS-0.5.tar.gz /usr/bin/perl Build.PL --installdirs site -- NOT OK Running Build test Make had some problems, won't test Running Build install Make had some problems, won't install Could not read '/root/.cpan/build/Net-Hadoop-WebHDFS-0.5-pJ80FW/META.yml'. Falling back to other methods to determine prerequisites CPAN: Module::Build loaded ok (v0.35)
結局エラーになったようで、
Could not read '/root/.cpan/build/Net-Hadoop-WebHDFS-0.5-pJ80FW/META.yml'
と出ていたので、中身を見てみる。
どうやら必要なモジュールなどがここに書かれているようだ。
# cat /root/.cpan/build/Net-Hadoop-WebHDFS-0.5-pJ80FW/META.yml --- abstract: 'Client library for Hadoop WebHDFS and HttpFs' author: - 'TAGOMORI Satoshi' build_requires: ExtUtils::MakeMaker: 6.36 Test::Deep: 0 Test::More: 0 configure_requires: CPAN::Meta: 0 CPAN::Meta::Prereqs: 0 ExtUtils::MakeMaker: 0 Module::Build: 0.38 dynamic_config: 0 generated_by: 'Minilla/v0.6.6, CPAN::Meta::Converter version 2.120921' license: perl meta-spec: url: http://module-build.sourceforge.net/META-spec-v1.4.html version: 1.4 name: Net-Hadoop-WebHDFS no_index: directory: - t - xt - inc - share - eg - examples - author provides: Net::Hadoop::WebHDFS: file: lib/Net/Hadoop/WebHDFS.pm version: 0.5 requires: Furl: 0 JSON: 0 JSON::XS: 0 Test::Deep: 0 Test::More: 0 Try::Tiny: 0 URI: 0 resources: bugtracker: https://github.com/tagomoris/Net-Hadoop-WebHDFS/issues homepage: https://github.com/tagomoris/Net-Hadoop-WebHDFS repository: git://github.com/tagomoris/Net-Hadoop-WebHDFS.git version: 0.5 x_contributors: - 'TAGOMORI Satoshi '
そしてエラーが出てはymlを見るってのを繰り返して、
最終的にこれだけのモジュールを入れたら、ちゃんと動くようになりました。
CPAN::Meta ExtUtils::MakeMaker Module::Build ExtUtils::CBuilder Test::Deep JSON JSON::XS Try::Tiny URI File::Temp Test::Requires ExtUtils::Constant Socket Test::SharedFork Time::HiRes Test::TCP Class::Accessor::Lite Encode HTTP::Parser::XS MIME::Base64 Mozilla::CA Scalar::Util Furl Net::Hadoop::WebHDFS
参考URL
http://d.hatena.ne.jp/perlcodesample/20100504/1270894115
http://tech.thekyo.jp/cpan%E3%83%A2%E3%82%B8%E3%83%A5%E3%83%BC%E3%83%AB%E3%80%8Cdatetime%E3%80%8D%E3%81%AE%E3%82%A4%E3%83%B3%E3%82%B9%E3%83%88%E3%83%BC%E3%83%AB/910/
http://deps.cpantesters.org/?module=Net%3A%3AHadoop%3A%3AWebHDFS&perl=5.20.0&os=any+OS
0 件のコメント:
コメントを投稿