README
1##########################################################################
2Copyright (c) 2009-2018, ETH Zurich.
3All rights reserved.
4
5This file is distributed under the terms in the attached LICENSE file.
6If you do not find this file, copies can be found by writing to:
7ETH Zurich D-INFK, Universitaetsstrasse 6, CH-8092 Zurich. Attn: Systems Group.
8##########################################################################
9
10Barrelfish Overview
11--------------------------------
12
13Barrelfish currently runs on:
14
15 * x86 CPUs in AMD64 mode. The following are known to work:
16
17 - Intel Xeon Clovertown, Gainestown, Beckton, IvyBridge, Haswell (X5355,
18 E5520, X7560, L5520, L7555, E5-2670v2, E3-1245v3)
19 - AMD Opteron Italy, Santa Rosa, Barcelona, Shanghai, Istanbul, Magny Cours
20 (275, 2220, 8350, 8374, 8380, 8431, 6174)
21 - Intel Xeon Phi Knights Corner
22 - QEMU simulator (2.2.10)
23
24 * ARM CPUs, specifically ARMv8 and ARMv7. The following platforms are known to work:
25
26 - ARMv7: The Texas Instruments OMAP4460 Pandaboard ES SoC
27 (Barrelfish runs on both the A9 and the M3 cores)
28 - ARMv7: The ARM VExpress EMM board as simulated by GEM5
29 - ARMv8: Cavium ThunderX2
30 - ARMv8: Applied Micro X-Gene
31 - ARMv8: NXP i.MX 8 SoC on a Toradex iMX8X Colibri board
32 - There is also limited support for the Netronome i8000 card, incorporating
33 a single Intel iXP2800 processor
34
35This README file provides instructions for the x86 architecture. For other
36architectures, please refer to the architecture-specific technical notes (e.g.,
37see TN 06 for the SCC, and TN 17 for ARM).
38
39You can either generate the latest documentation from this source-code
40(instructions at end of this file), or visit the Barrelfish_ website
41to download them.
42
43.. _Barrelfish: http://www.barrelfish.org/
44
45Supported PC hardware
46--------------------------------
47
48The biggest compatibility problems are likely to be in the PCI/ACPI code. We
49usually discover new quirks (or missing functionality in the ACPI glue code)
50on each new machine we test. The following systems are known to work:
51
52 * Intel x5000XVN
53 * Tyan n6650W and S4985
54 * Supermicro H8QM3-2
55 * Dell PowerEdge R610 and R905
56 * Sun Fire X2270 and X4440
57 * Intel/Quanta QSSC-S4R
58 * Lenovo X200 and X301 laptops
59 * ASUS Eee PC 1015PEM netbooks
60
61We have support for the following NICs:
62 * The e1000n should work with most recent Intel gigabit ethernet
63 controllers (see the list in devices/e1000.dev). We've mostly used the
64 82572EI (PCI device ID 0x1082).
65 * Intel i82599 (or X520) 10GbE
66 * Solarflare sfn5122f 10GbE
67
68You should also be able to boot Barrelfish on a recent version of QEMU (2.14);
69note that the e1000 device emulated by QEMU is not supported by our driver.
70
71Required Tools
72--------------------------------
73
74Our toolchain tracks Ubuntu LTS releases, and this is what we use to
75run our nightly tests. If you are running Ubuntu LTS (18.04.1 at time
76of writing), this means the following:
77
78 * GCC 7.4.0 for x86_64, ARMv7 and ARMv8
79 - On Ubuntu LTS install packages:
80 build-essential g++
81 gcc-arm-linux-gnueabi g++-arm-linux-gnueabi
82 gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
83 * GNU binutils (2.30 is supported)
84 * GNU make
85 * GHC v8.0.2 and Parsec 3.1
86 - On Ubuntu LTS, install the following packages:
87 cabal-install libghc-src-exts-dev libghc-ghc-paths-dev
88 libghc-parsec3-dev libghc-random-dev libghc-ghc-mtl-dev
89 libghc-async-dev libghc-aeson-pretty-dev libghc-aeson-dev
90 libghc-missingh-dev
91 - Then, run cabal update && cabal install bytestring-trie pretty-simple
92 * FreeBSD's libelf:
93 - On Ubuntu install the following packages:
94 libelf-freebsd-dev freebsd-glue
95 * Optional: LibUSB 1.0 (for the usbboot tool):
96 - On Ubuntu install libusb-1.0-0-dev
97
98 * Optional: QEMU with e1000e EFI ROM:
99 - On Ubuntu install qemu-system-x86 qemu-efi qemu-ipxe
100 - Then cd /usr/lib/ipxe/qemu/ &&
101 sudo wget https://github.com/qemu/qemu/raw/master/pc-bios/efi-e1000e.rom
102
103 * Optional ARMv8 support:
104 - prefix all deb entries in /etc/apt/sources.list with [arch=amd64]
105 - add "deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports bionic main universe"
106 to sources.list
107 - dpkg --add-architecture arm64 && apt update && apt install gnu-efi:arm64 libefiboot-dev
108
109Building
110--------------------------------
111
1121. Assuming you have already unpacked the sources, create a build directory ::
113
114 $ mkdir build && cd build
115
1161. Run ``hake.sh``, giving it the path to the source directory and target
117architecture(s) ::
118
119 $ ../hake/hake.sh -s ../ -a x86_64
120
121This will configure the build directory and use GHC to compile and then run
122hake, a tool used to generate the ``Makefile``.
123
1243. Optionally, edit the configuration parameters in ``hake/Config.hs`` and
125run ``make rehake`` to apply them.
126
1274. Run make, and wait ::
128
129 $ make X86_64_Basic
130
1315. If everything worked, you should now be able to run Barrelfish inside QEMU
132 (make sure you have followed the optional qemu step in required tools) ::
133
134 $ make qemu_x86_64
135
136Installing and Booting
137--------------------------------
138
139Barrelfish requires a Multiboot-compliant bootloader that is capable of loading
140an ELF64 image. At the time of writing, this doesn't include the default GRUB.
141Your options are either:
142
143 * use the pre-loader "elver" that can be found in the tools directory
144 * patch GRUB to support a 64-bit kernel image, using this patch_.
145
146.. _patch: http://savannah.gnu.org/bugs/?17963
147
148"Installing" Barrelfish currently consists of copying the ELF files for the CPU
149driver and user programs to a location that the target machine can boot from,
150and writing a suitable menu.lst file that instructs the bootloader (GRUB) which
151programs to load and the arguments to pass them.
152
153If you specify an appropriate INSTALL_PREFIX, ``make install`` will copy the
154binaries to the right place for you, eg ::
155
156 $ make install INSTALL_PREFIX=/tftpboot/barrelfish
157
158We usually boot Barrelfish via PXE/TFTP, although loading from a local disk
159also works. Instructions for setting up GRUB to do this are beyond the scope of
160this document. Assuming you have such a setup, a sample menu.lst can
161be found in build/x86/menu.lst.x86_64 after executin make qemu_x86_64. It
162is a basic diskless boot that doesn't do anything useful beyond probing the
163PCI buses and starting a basic shell.
164
165There are many other programs you can load (take a look around the usr tree for
166examples). To start a program on a core other than the BSP core, pass
167``core=N`` as its first argument.
168
169If things work, you should see output on both the VGA console and COM1.
170
171Generating Documentation
172--------------------------------
173
174Barrelfish documentation can be found on Barrelfish website
175(http://www.barrelfish.org/). And it can be also generated from the code tree.
176For documentation generation, you will need ``latex`` packages installed,
177including support for ``pdflatex``. Following are the instructions for
178generating the documentation assuming you have already unpacked the sources ::
179
180 $ mkdir build && cd build
181 $ ../hake/hake.sh -s ../
182 $ make docs
183
184You will find all the technotes in ``docs/`` directory.
185
186Known Issues
187--------------------------------
188
189There are many. Those you're likely to encounter include:
190
191 * The documentation is incomplete and out of date.
192 * Some drivers and user programs are known not to build, and are
193 not included in the default set of targets (MODULES) in the Makefile.
194
195Likely FAQs
196--------------------------------
197
198Q: How do I run a program?
199A: Add it to the boot sequence by specifying the module in your menu.lst file.
200 For example, to run the memtest program, add the line:
201 module /PATH/x86_64/sbin/memtest
202 to the end of menu.lst, where PATH is relative either to your TFTP
203 server's root directory (when booting on hardware) or to your build
204 directory (when using a simulator such as QEMU).
205 If memtest runs, you should see it output "memtest passed successfully!".
206
207Q: How do develop a program?
208A: Check out Technote 18, Practical Guide. Either on barrelfish.org or
209 in the generated technotes (see Generating Documentation)
210
211
212Q: Where's the CPU driver?
213A: It's in the directory named kernel :) But don't worry, it really does run
214 independently on each core.
215
216Q: Where is the source for the SPLASH2 benchmarks? It seems to be missing.
217A: The license for these prevents redistribution, so we were forced to ship our
218 changes as a patch. See usr/splash2/README for further instructions.
219
220Q: Can I use a debugger?
221A: Maybe. There are two options at the moment:
222 * On a simulator, using whatever debug interfaces it supports.
223 For QEMU, you could try the "debugsim" target.
224 * On hardware, using the kernel-mode remote GDB stubs that operate on the
225 primary serial port and are entered in response to a kernel trap or
226 exception. However, these are not well maintained, and may not be usable
227 beyond reading/writing memory locations and inspecting the stack.
228 When debugging the kernel, beware that it is relocated to an address
229 determined at core boot time. Look for output such as:
230 "Kernel starting at address 0xffffffffc072b000".
231
232Q: Where can I find more information, including papers and new releases?
233A: http://www.barrelfish.org/
234 http://wiki.barrelfish.org/
235
236Q: Can I contribute?
237A: We'd certainly like to hear from you. Feel free to send patches (or even
238 git merge requests) to the Barrelfish mailing list.
239
240 To keep track of contributions to Barrelfish, we use a sign-off procedure
241 similar to the Linux kernel:
242
243 The sign-off is a simple line at the end of the explanation for the patch,
244 which certifies that you wrote it or otherwise have the right to pass it on
245 as an open-source patch. The rules are pretty simple: if you can certify
246 the below:
247
248 Developer's Certificate of Origin 1.1
249
250 By making a contribution to this project, I certify that:
251
252 (a) The contribution was created in whole or in part by me and I
253 have the right to submit it under the open source license
254 indicated in the file; or
255
256 (b) The contribution is based upon previous work that, to the best
257 of my knowledge, is covered under an appropriate open source
258 license and I have the right under that license to submit that
259 work with modifications, whether created in whole or in part
260 by me, under the same open source license (unless I am
261 permitted to submit under a different license), as indicated
262 in the file; or
263
264 (c) The contribution was provided directly to me by some other
265 person who certified (a), (b) or (c) and I have not modified
266 it.
267
268 (d) I understand and agree that this project and the contribution
269 are public and that a record of the contribution (including all
270 personal information I submit with it, including my sign-off) is
271 maintained indefinitely and may be redistributed consistent with
272 this project or the open source license(s) involved.
273
274 then you just add a line saying
275
276 Signed-off-by: Random J Developer <random@developer.example.org>
277
278 Note that git has support for adding such a message in the end of the commit
279 log message.
280
README_ARRAKIS
1##########################################################################
2Copyright (c) 2013-2014, University of Washington.
3All rights reserved.
4
5This file is distributed under the terms in the attached LICENSE file.
6If you do not find this file, copies can be found by writing to:
7ETH Zurich D-INFK, Universitaetstrasse 6, CH-8092 Zurich. Attn: Systems Group.
8##########################################################################
9
10Arrakis README Supplement
11--------------------------------
12
13This version of Arrakis is a fork of Barrelfish release
142014-03-11. Arrakis contains many additions and fixes to the vanilla
15Barrelfish tree, notably:
16
17 * Boot directly via QEMU Multiboot support, instead of GRUB
18 * Arrakis domains: These run in guest ring 0, hardware-virtualized
19 * Parallel hake
20 * SR-IOV support
21 * Fixes to PCI bus enumeration
22 * Can use BIOS preset values to configure PCI (like Linux)
23 * Driver for 82599 virtual function
24 * More POSIX support, in particular pthreads, epoll, and sockets
25 * Intel performance monitoring counters support
26 * Arranet, the Arrakis network stack
27 * Support for advanced 82599 features, like weighted round-robin
28 scheduling and rate limiting
29 * TenaciousD, a persistent data structure library
30 * libstorage, a storage HAL
31 * Intel MegaRAID device driver
32 * Intel VT-d (IOMMU) driver
33
34Arrakis likely also contains bugs not present in Barrelfish. In some
35cases, there is likely to be some debug code left in this release,
36which might impact your experience. Don't despair! Just comment out
37suspicious-looking code and see if it makes Arrakis work for
38you. Unfortunately, we do not have the man or machine power to test
39Arrakis as extensively as the Barrelfish releases.
40
41We make an effort to back-port our changes to Barrelfish periodically,
42by which time they will have stabilized and tested on the broad range
43of machines that Barrelfish is usually expected to run on.
44
45.. _Arrakis: http://arrakis.cs.washington.edu/
46
47Supported PC hardware
48--------------------------------
49
50Arrakis has been tested on the following PC hardware:
51
52 * Dell PowerEdge R520 servers in x86-64 mode. Our server consists of
53 a 6-core Intel Xeon E5-2430 (Sandy Bridge) system at 2.2 GHz clock
54 frequency with 4GB of RAM. The tested peripherals are:
55
56 - Intel X520 dual-port 10Gb Ethernet adapter
57 - Intel MegaRAID RS3DC040 RAID controller
58
59Required Tools
60--------------------------------
61
62In addition to the tools required by Barrelfish, Arrakis requires the following:
63
64 * GHC Control.Parallel.Strategies library
65
66Building
67--------------------------------
68
69The build process (including required tools) is unchanged from
70Barrelfish. Please refer to the README file for instructions.
71
72Installing and Booting
73--------------------------------
74
75Installation and booting is also unchanged from Barrelfish and
76explained in the README file.
77
78A number of commandline options to system daemons and device drivers
79have been added:
80
81 * pci now supports an "skb_bridge_program=" option, to set which SKB
82 program to use to configure PCI bridges. "bridge_bios" will use the
83 BIOS preset values and might be preferable on some systems.
84
85 * pci now supports the "numvfs=" option to set the number of virtual
86 functions to configure (for every device that supports them).
87
88 * VT-d related commandline options have been added and are explained
89 in usr/acpi/README_VTD.
90
91 * The 82599 physical function driver supports a range of options to
92 configure advanced NIC features. "tx_rate[x]=y" is used to limit
93 the transmit rate of virtual function x to y
94 Mbps. credit_refill[x]=y" is used to set the number of credits for
95 weighted round-robin scheduling of queue x to y.
96
97 * The 82599 virtual function driver accepts an option "vf=x" to
98 configure the virtual function number to use.
99
README_NETWORKING
1--------------------------------------------
2This file describes how to use and how to hack networking subsystem of
3the Barrelfish.
4
5--------------------------------------------
6Using networking subsystem:
7Currently Barrelfish supports following NIC devices
8rtl8029 (Realtek RTL8029AS, emulation support QEMU)
9e1000 (Intel 82576 1GbE NIC)
10e10k (Intel 82599 10GbE NIC)
11eMAC (on SCC)
12
13If you want to use any of above then you should edit the menu.lst file to
14reflect which card you want to use. The following example shows how to
15use e1000 card.
16
17Add following lines to menu.lst
18{{{
19module /x86_64/sbin/e1000n
20module /x86_64/sbin/NGD_mng cardname=e1000
21module /x86_64/sbin/netd cardname=e1000
22}}}
23
24e1000n: The first line will start a driver domain which will be responsible for
25sending and receiving packets on one queue within the NIC.
26
27NGD_mng: The second line starts the "net device manager" service which is
28responsible for port management and handling of hardware queues present within
29NIC (if any). The "cardname=e1000" argument tells this "net device manager"
30that it is responsible for NIC card e1000.
31
32netd: The third line starts the netd service is responsible for doing DHCP
33to get an IP for the NIC, running ARP lookups on behalf of all other
34applications, and handling the traffic which no other application wants.
35This daemon also has an argument stating which card it will be responsible for.
36
37
38--------------------------------------------
39How to use networking from within application?
40Every application that wants to use the networking should initialize the
41networking. It can be done by calling a following function.
42{{{
43lwip_init(char *card_name, uint64_t qid)
44}}}
45Here, card_name is the name of the NIC device to be used (eg: e1000), and
46queue_id the id of the hardware queue that should be used. The queue_id 0
47is always safe option, and queue 0 is assumed to be a shared queue.
48Following is the example call:
49{{{
50lwip_init("e1000", 0);
51}}}
52
53One of the variant of this function "lwip_init_auto" tries to guess the
54NIC device name automatically by some rudimentary logic, but it does not
55yet work for all devices and in all situations.
56
57Once "lwip_init" returns successfully, the application can start using the
58networking. The networking interface is based on the "Light Weight
59Internet Protocol (LWIP)". Also applications like "webserver",
60"echoserver", "vmit" use networking, and hence can be referred to
61check how to use the Barrelfish networking.
62
63--------------------------------------------
64Known limitations:
65The presence of DHCP server within same LAN is assumed to give an IP
66address.
67
68Currently, only one NIC device can be effectively used in one running
69instance of Barrelfish.
70The reason behind this is that currently, netd is stateless. Even though most
71of the networking code can work even in presence of multiple active NIC
72devices of different type, netd is still not completely ready for this
73possibility.
74
75--------------------------------------------
76
77
78