Package Details: hadoop 3.4.1-1

Git Clone URL: https://aur.archlinux.org/hadoop.git (read-only, click to copy)
Package Base: hadoop
Description: An open-source software for reliable, scalable, distributed computing
Upstream URL: https://hadoop.apache.org/
Keywords: apache big-data java map-reduce
Licenses: Apache
Conflicts: yarn
Submitter: sjakub
Maintainer: matthewq337
Last Packager: matthewq337
Votes: 85
Popularity: 0.24
First Submitted: 2009-04-07 16:39 (UTC)
Last Updated: 2024-11-11 04:00 (UTC)

Dependencies (10)

Required by (3)

Sources (8)

Latest Comments

« First ‹ Previous 1 2 3 4 5 6 7 8 Next › Last »

Musikolo commented on 2020-04-12 16:39 (UTC)

Hi @qsdrqs,

I don't know how to help you with your question about Yarn, but if you want to find the systemd services available, you can do as follows:

[musikolo@MyPc ~]$ pacman -Ql hadoop | grep 'service$'
hadoop /usr/lib/systemd/system/hadoop-datanode.service
hadoop /usr/lib/systemd/system/hadoop-jobtracker.service
hadoop /usr/lib/systemd/system/hadoop-namenode.service
hadoop /usr/lib/systemd/system/hadoop-secondarynamenode.service
hadoop /usr/lib/systemd/system/hadoop-tasktracker.service

I hope it helps.

qsdrqs commented on 2020-04-01 05:57 (UTC) (edited on 2020-04-01 05:57 (UTC) by qsdrqs)

Hello! How can I start yarn service through this package, the script in hadoop/sbin may not recognize my config in /etc, and I can't find any systemd service on my computer to start it.

Looking forward to you reply!

takaomag commented on 2019-12-04 08:33 (UTC) (edited on 2019-12-04 08:34 (UTC) by takaomag)

When I installed this package by yay, I received the following message in the terminal.

yay -S --needed --noconfirm --noprogressbar hadoop

...

==> Removing existing $srcdir/ directory...

==> Extracting sources...

-> Extracting hadoop-3.2.1.tar.gz with bsdtar

==> Sources are ready.

removing Untracked AUR files from cache...

:: Cleaning (1/1): /var/lib/x-aur-helper/.cache/yay/hadoop

Removing hadoop-3.2.1.tar.gz

Can not find package name : [apache.osuosl.org]

I did not modify the PKGBUILD. Does someone knows any solution?

dxxvi commented on 2017-06-07 05:08 (UTC) (edited on 2017-06-07 06:07 (UTC) by dxxvi)

How do I start this hadoop? I try: sudo systemctl start hadoop-datanode hadoop-jobtracker hadoop-namenode hadoop-secondarynamenode hadoop-tasktracker then check their status: systemctl status hadoop-datanode hadoop-jobtracker hadoop-namenode hadoop-secondarynamenode hadoop-tasktracker All of them failed. The jobtracker has this line: Error: JAVA_HOME is not set and could not be found. ------------------------------------------------------------------------------ Self-answer: JAVA_HOME error: https://wiki.archlinux.org/index.php/Hadoop Unable to start namenode and datanode: Hadoop ArchWiki to format a new distributed filesystem; https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-common/SingleCluster.html for editing core-site.xml and hdfs-site.xml jobtracker and tasktracker cannot start: running the commands in hadoop-jobtracker.service and hadoop-tasktracker.service under the hadoop account shows the reasons (12eason also mentioned that).

12eason commented on 2017-03-14 22:45 (UTC)

First thing, hdfs, mapred, container-executor, rcc and yarn all need to be linked to /usr/bin along with hadoop. Hdfs especially has a lot of the functions previously done by hadoop. Secondly, the hadoop package provides shell scripts under sbin/ to start and stop instances and these would be less prone to breakage if used in the systemd scripts. As it is, many commands systemd uses are depreciated.

nmiculinic commented on 2017-03-11 17:47 (UTC)

There's mirror problems for hadoop: ==> Making package: hadoop 2.7.3-1 (Sat Mar 11 18:48:07 CET 2017) ==> Retrieving sources... -> Downloading hadoop-2.7.3.tar.gz... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 Warning: Transient problem: HTTP error Will retry in 3 seconds. 3 retries Warning: left.

flipflop97 commented on 2016-11-21 13:36 (UTC)

Can you symlink /usr/lib/hadoop/bin/mapred to /usr/bin/mapred

severach commented on 2016-09-13 19:07 (UTC) (edited on 2016-09-13 19:18 (UTC) by severach)

I'm looking to save time for others, not myself. The problem is that xz is very useful in the repos where traffic reduction is worth any cost. xz is counter productive on the AUR.

petronny commented on 2016-09-13 03:40 (UTC)

Hi, I found your discussion about the PKGEXT. But have you ever tried to compress the package in parallel?(by setup 'xz -T0' in /etc/makepkg.conf) I got .pkg.tar.xz: 530% cpu 26.731s with CPU E5-2660 0 @ 2.20GHz And I think it may take much less time on a i3/5/7 cpu

ael commented on 2016-07-18 08:47 (UTC)

`hadoop-jobtracker.service` make use of command `/usr/bin/hadoop jobtracker` but is deprecated. The ouput of that command suggest to use the new yarn command.

« First ‹ Previous 1 2 3 4 5 6 7 8 Next › Last »