1##########################################################################
2Copyright (c) 2009-2016, ETH Zurich.
3All rights reserved.
4
5This file is distributed under the terms in the attached LICENSE file.
6If you do not find this file, copies can be found by writing to:
7ETH Zurich D-INFK, Universitaetsstrasse 6, CH-8092 Zurich. Attn: Systems Group.
8##########################################################################
9
10Barrelfish Overview
11--------------------------------
12
13Barrelfish currently runs on:
14
15 * x86 CPUs in either IA-32 or AMD64 mode. The following are known to work:
16
17 - Intel Xeon Clovertown, Gainestown, Beckton, IvyBridge, Haswell (X5355,
18 E5520, X7560, L5520, L7555, E5-2670v2, E3-1245v3)
19 - AMD Opteron Italy, Santa Rosa, Barcelona, Shanghai, Istanbul, Magny Cours
20 (275, 2220, 8350, 8374, 8380, 8431, 6174)
21 - QEMU simulator (2.0.0)
22
23 * ARM CPUs, specifically ARMv7 and ARMv5. The following platforms are known to work:
24
25 - The integratorcp ARMv5 machine as simulated by QEM
26 - The Texas Instruments OMAP4460 Pandaboard ES SoC
27 (Barrelfish runs on both the A9 and the M3 cores)
28 - The ARM VExpress EMM board as simulated by GEM5
29 - There is also limited support for the Netronome i8000 card, incorporating
30 a single Intel iXP2800 processor
31
32This README file provides instructions for the x86 architecture. For other
33architectures, please refer to the architecture-specific technical notes (e.g.,
34see TN 06 for the SCC, and TN 17 for ARM).
35
36You can either generate the latest documentation from this source-code
37(instructions at end of this file), or visit the Barrelfish_ website
38to download them.
39
40.. _Barrelfish: http://www.barrelfish.org/
41
42Supported PC hardware
43--------------------------------
44
45Barrelfish supports following PC hardware :
46
47 * x86 CPUs in either IA-32 or AMD64 mode. The following are known to work:
48
49 - Intel Xeon Clovertown, Gainestown, Beckton, Ivy Bridge, Haswell (X5355,
50 E5520, X7560, L5520, L7555, E5-2670v2, E3-1245v3)
51 - AMD Opteron Italy, Santa Rosa, Barcelona, Shanghai, Istanbul, Magny Cours
52 (275, 2220, 8350, 8374, 8380, 8431, 6174)
53 * Note: the Opteron 275 do not work in IA-32 mode.
54
55The biggest compatibility problems are likely to be in the PCI/ACPI code. We
56usually discover new quirks (or missing functionality in the ACPI glue code)
57on each new machine we test. The following systems are known to work:
58
59 * Intel x5000XVN
60 * Tyan n6650W and S4985
61 * Supermicro H8QM3-2
62 * Dell PowerEdge R610 and R905
63 * Sun Fire X2270 and X4440
64 * Intel/Quanta QSSC-S4R
65 * Lenovo X200 and X301 laptops
66 * ASUS Eee PC 1015PEM netbooks
67
68The e1000n driver should work with most recent Intel gigabit ethernet
69controllers (see the list in devices/e1000.dev). We've mostly used the
7082572EI (PCI device ID 0x1082).
71
72You should also be able to boot Barrelfish on a recent version of QEMU (0.14);
73note that the e1000 device emulated by QEMU is not supported by our driver.
74
75Required Tools
76--------------------------------
77
78Our toolchain tracks Ubuntu LTS releases, and this is what we use to
79run our nightly tests. If you are running Ubuntu LTS (16.04.1 at time
80of writing), this means the following:
81
82 * GCC 5.4.0 for x86_64, ARMv7 and ARMv8
83 - Install packages gcc-arm-linux-gnueabi, g++-arm-linux-gnueabi,
84 gcc-aarch64-linux-gnu and g++-aarch64-linux-gnu on Ubuntu LTS
85 * GNU binutils (2.24 and 2.26 are known to work)
86 * GNU make
87 * GHC v7.10.3 and Parsec 3.1
88 - older versions of the tree supported v6.10 or v6.12.2 with Parsec
89 2.1 or v7.4 with Parsec 3.1, or v7.6.3 with Parsec 3.1; GHC v6.12.1 has a
90 known bug and is unable to build our tools, but now earlier versions of
91 GHC are unsupported.
92 - On Ubuntu LTS, install the following packages:
93 libghc-mtl-dev libghc-ghc-mtl-dev cabal-install libghc-src-exts-dev
94 libghc-async-dev libghc-ghc-paths-dev libghc-parsec3-dev
95 libghc-random-dev
96 - Then, run cabal-install bytestring-trie
97 * FreeBSD's libelf:
98 - On Ubuntu install the following packages:
99 libelf-freebsd-dev freebsd-glue
100 * LibUSB 1.0 (for the usbboot tool):
101 - On Ubuntu install libusb-1.0-0-dev
102
103Building
104--------------------------------
105
1061. Assuming you have already unpacked the sources, create a build directory ::
107
108 $ mkdir build && cd build
109
1101. Run ``hake.sh``, giving it the path to the source directory and target
111architecture(s) ::
112
113 $ ../hake/hake.sh -s ../ -a x86_64
114
115This will configure the build directory and use GHC to compile and then run
116hake, a tool used to generate the ``Makefile``.
117
1183. Optionally, edit the configuration parameters in ``hake/Config.hs`` and
119run ``make rehake`` to apply them.
120
1214. Run make, and wait ::
122
123 $ make X86_64_Basic
124
1255. If everything worked, you should now be able to run Barrelfish inside QEMU ::
126
127 $ make qemu_x86_64
128
129Installing and Booting
130--------------------------------
131
132Barrelfish requires a Multiboot-compliant bootloader that is capable of loading
133an ELF64 image. At the time of writing, this doesn't include the default GRUB.
134Your options are either:
135
136 * use the pre-loader "elver" that can be found in the tools directory
137 * patch GRUB to support a 64-bit kernel image, using this patch_.
138
139.. _patch: http://savannah.gnu.org/bugs/?17963
140
141"Installing" Barrelfish currently consists of copying the ELF files for the CPU
142driver and user programs to a location that the target machine can boot from,
143and writing a suitable menu.lst file that instructs the bootloader (GRUB) which
144programs to load and the arguments to pass them.
145
146If you specify an appropriate INSTALL_PREFIX, ``make install`` will copy the
147binaries to the right place for you, eg ::
148
149 $ make install INSTALL_PREFIX=/tftpboot/barrelfish
150
151We usually boot Barrelfish via PXE/TFTP, although loading from a local disk
152also works. Instructions for setting up GRUB to do this are beyond the scope of
153this document. Assuming you have such a setup, here is a sample menu.lst file
154for a basic diskless boot that doesn't do anything useful beyond probing the
155PCI buses and starting a basic shell ::
156
157 title Barrelfish
158 root (nd)
159 kernel /barrelfish/x86_64/sbin/elver
160 module /barrelfish/x86_64/sbin/cpu
161 module /barrelfish/x86_64/sbin/init
162 module /barrelfish/x86_64/sbin/mem_serv
163 module /barrelfish/x86_64/sbin/monitor
164 module /barrelfish/x86_64/sbin/ramfsd boot
165 module /barrelfish/x86_64/sbin/skb boot
166 modulenounzip /barrelfish/skb_ramfs.cpio.gz nospawn
167 module /barrelfish/x86_64/sbin/acpi boot
168 module /barrelfish/x86_64/sbin/pci boot
169 module /barrelfish/x86_64/sbin/spawnd boot
170 module /barrelfish/x86_64/sbin/serial
171 module /barrelfish/x86_64/sbin/fish
172
173There are many other programs you can load (take a look around the usr tree for
174examples). To start a program on a core other than the BSP core, pass
175``core=N`` as its first argument.
176
177If things work, you should see output on both the VGA console and COM1.
178
179Generating Documentation
180--------------------------------
181
182Barrelfish documentation can be found on Barrelfish website
183(http://www.barrelfish.org/). And it can be also generated from the code tree.
184For documentation generation, you will need ``latex`` packages installed,
185including support for ``pdflatex``. Following are the instructions for
186generating the documentation assuming you have already unpacked the sources ::
187
188 $ mkdir build && cd build
189 $ ../hake/hake.sh -s ../
190 $ make docs
191
192You will find all the technotes in ``docs/`` directory.
193
194Known Issues
195--------------------------------
196
197There are many. Those you're likely to encounter include:
198
199 * The documentation is incomplete and out of date.
200 * Some drivers and user programs are known not to build, and are
201 not included in the default set of targets (MODULES) in the Makefile.
202
203Likely FAQs
204--------------------------------
205
206Q: How do I run a program?
207A: Add it to the boot sequence by specifying the module in your menu.lst file.
208 For example, to run the memtest program, add the line:
209 module /PATH/x86_64/sbin/memtest
210 to the end of menu.lst, where PATH is relative either to your TFTP
211 server's root directory (when booting on hardware) or to your build
212 directory (when using a simulator such as QEMU).
213 If memtest runs, you should see it output "memtest passed successfully!".
214
215Q: Where's the CPU driver?
216A: It's in the directory named kernel :) But don't worry, it really does run
217 independently on each core.
218
219Q: Where is the source for the SPLASH2 benchmarks? It seems to be missing.
220A: The license for these prevents redistribution, so we were forced to ship our
221 changes as a patch. See usr/splash2/README for further instructions.
222
223Q: Can I use a debugger?
224A: Maybe. There are two options at the moment:
225 * On a simulator, using whatever debug interfaces it supports.
226 For QEMU, you could try the "debugsim" target.
227 * On hardware, using the kernel-mode remote GDB stubs that operate on the
228 primary serial port and are entered in response to a kernel trap or
229 exception. However, these are not well maintained, and may not be usable
230 beyond reading/writing memory locations and inspecting the stack.
231 When debugging the kernel, beware that it is relocated to an address
232 determined at core boot time. Look for output such as:
233 "Kernel starting at address 0xffffffffc072b000".
234
235Q: Where can I find more information, including papers and new releases?
236A: http://www.barrelfish.org/
237 http://wiki.barrelfish.org/
238
239Q: Can I contribute?
240A: We'd certainly like to hear from you. Feel free to send patches (or even
241 git merge requests) to the Barrelfish mailing list.
242
243 To keep track of contributions to Barrelfish, we use a sign-off procedure
244 similar to the Linux kernel:
245
246 The sign-off is a simple line at the end of the explanation for the patch,
247 which certifies that you wrote it or otherwise have the right to pass it on
248 as an open-source patch. The rules are pretty simple: if you can certify
249 the below:
250
251 Developer's Certificate of Origin 1.1
252
253 By making a contribution to this project, I certify that:
254
255 (a) The contribution was created in whole or in part by me and I
256 have the right to submit it under the open source license
257 indicated in the file; or
258
259 (b) The contribution is based upon previous work that, to the best
260 of my knowledge, is covered under an appropriate open source
261 license and I have the right under that license to submit that
262 work with modifications, whether created in whole or in part
263 by me, under the same open source license (unless I am
264 permitted to submit under a different license), as indicated
265 in the file; or
266
267 (c) The contribution was provided directly to me by some other
268 person who certified (a), (b) or (c) and I have not modified
269 it.
270
271 (d) I understand and agree that this project and the contribution
272 are public and that a record of the contribution (including all
273 personal information I submit with it, including my sign-off) is
274 maintained indefinitely and may be redistributed consistent with
275 this project or the open source license(s) involved.
276
277 then you just add a line saying
278
279 Signed-off-by: Random J Developer <random@developer.example.org>
280
281 Note that git has support for adding such a message in the end of the commit
282 log message.
283
1##########################################################################
2Copyright (c) 2013-2014, University of Washington.
3All rights reserved.
4
5This file is distributed under the terms in the attached LICENSE file.
6If you do not find this file, copies can be found by writing to:
7ETH Zurich D-INFK, Haldeneggsteig 4, CH-8092 Zurich. Attn: Systems Group.
8##########################################################################
9
10Arrakis README Supplement
11--------------------------------
12
13This version of Arrakis is a fork of Barrelfish release
142014-03-11. Arrakis contains many additions and fixes to the vanilla
15Barrelfish tree, notably:
16
17 * Boot directly via QEMU Multiboot support, instead of GRUB
18 * Arrakis domains: These run in guest ring 0, hardware-virtualized
19 * Parallel hake
20 * SR-IOV support
21 * Fixes to PCI bus enumeration
22 * Can use BIOS preset values to configure PCI (like Linux)
23 * Driver for 82599 virtual function
24 * More POSIX support, in particular pthreads, epoll, and sockets
25 * Intel performance monitoring counters support
26 * Arranet, the Arrakis network stack
27 * Support for advanced 82599 features, like weighted round-robin
28 scheduling and rate limiting
29 * TenaciousD, a persistent data structure library
30 * libstorage, a storage HAL
31 * Intel MegaRAID device driver
32 * Intel VT-d (IOMMU) driver
33
34Arrakis likely also contains bugs not present in Barrelfish. In some
35cases, there is likely to be some debug code left in this release,
36which might impact your experience. Don't despair! Just comment out
37suspicious-looking code and see if it makes Arrakis work for
38you. Unfortunately, we do not have the man or machine power to test
39Arrakis as extensively as the Barrelfish releases.
40
41We make an effort to back-port our changes to Barrelfish periodically,
42by which time they will have stabilized and tested on the broad range
43of machines that Barrelfish is usually expected to run on.
44
45.. _Arrakis: http://arrakis.cs.washington.edu/
46
47Supported PC hardware
48--------------------------------
49
50Arrakis has been tested on the following PC hardware:
51
52 * Dell PowerEdge R520 servers in x86-64 mode. Our server consists of
53 a 6-core Intel Xeon E5-2430 (Sandy Bridge) system at 2.2 GHz clock
54 frequency with 4GB of RAM. The tested peripherals are:
55
56 - Intel X520 dual-port 10Gb Ethernet adapter
57 - Intel MegaRAID RS3DC040 RAID controller
58
59Required Tools
60--------------------------------
61
62In addition to the tools required by Barrelfish, Arrakis requires the following:
63
64 * GHC Control.Parallel.Strategies library
65
66Building
67--------------------------------
68
69The build process (including required tools) is unchanged from
70Barrelfish. Please refer to the README file for instructions.
71
72Installing and Booting
73--------------------------------
74
75Installation and booting is also unchanged from Barrelfish and
76explained in the README file.
77
78A number of commandline options to system daemons and device drivers
79have been added:
80
81 * pci now supports an "skb_bridge_program=" option, to set which SKB
82 program to use to configure PCI bridges. "bridge_bios" will use the
83 BIOS preset values and might be preferable on some systems.
84
85 * pci now supports the "numvfs=" option to set the number of virtual
86 functions to configure (for every device that supports them).
87
88 * VT-d related commandline options have been added and are explained
89 in usr/acpi/README_VTD.
90
91 * The 82599 physical function driver supports a range of options to
92 configure advanced NIC features. "tx_rate[x]=y" is used to limit
93 the transmit rate of virtual function x to y
94 Mbps. credit_refill[x]=y" is used to set the number of credits for
95 weighted round-robin scheduling of queue x to y.
96
97 * The 82599 virtual function driver accepts an option "vf=x" to
98 configure the virtual function number to use.
99
1--------------------------------------------
2This file describes how to use and how to hack networking subsystem of
3the Barrelfish.
4
5--------------------------------------------
6Using networking subsystem:
7Currently Barrelfish supports following NIC devices
8rtl8029 (Realtek RTL8029AS, emulation support QEMU)
9e1000 (Intel 82576 1GbE NIC)
10e10k (Intel 82599 10GbE NIC)
11eMAC (on SCC)
12
13If you want to use any of above then you should edit the menu.lst file to
14reflect which card you want to use. The following example shows how to
15use e1000 card.
16
17Add following lines to menu.lst
18{{{
19module /x86_64/sbin/e1000n
20module /x86_64/sbin/NGD_mng cardname=e1000
21module /x86_64/sbin/netd cardname=e1000
22}}}
23
24e1000n: The first line will start a driver domain which will be responsible for
25sending and receiving packets on one queue within the NIC.
26
27NGD_mng: The second line starts the "net device manager" service which is
28responsible for port management and handling of hardware queues present within
29NIC (if any). The "cardname=e1000" argument tells this "net device manager"
30that it is responsible for NIC card e1000.
31
32netd: The third line starts the netd service is responsible for doing DHCP
33to get an IP for the NIC, running ARP lookups on behalf of all other
34applications, and handling the traffic which no other application wants.
35This daemon also has an argument stating which card it will be responsible for.
36
37
38--------------------------------------------
39How to use networking from within application?
40Every application that wants to use the networking should initialize the
41networking. It can be done by calling a following function.
42{{{
43lwip_init(char *card_name, uint64_t qid)
44}}}
45Here, card_name is the name of the NIC device to be used (eg: e1000), and
46queue_id the id of the hardware queue that should be used. The queue_id 0
47is always safe option, and queue 0 is assumed to be a shared queue.
48Following is the example call:
49{{{
50lwip_init("e1000", 0);
51}}}
52
53One of the variant of this function "lwip_init_auto" tries to guess the
54NIC device name automatically by some rudimentary logic, but it does not
55yet work for all devices and in all situations.
56
57Once "lwip_init" returns successfully, the application can start using the
58networking. The networking interface is based on the "Light Weight
59Internet Protocol (LWIP)". Also applications like "webserver",
60"echoserver", "vmit" use networking, and hence can be referred to
61check how to use the Barrelfish networking.
62
63--------------------------------------------
64Known limitations:
65The presence of DHCP server within same LAN is assumed to give an IP
66address.
67
68Currently, only one NIC device can be effectively used in one running
69instance of Barrelfish.
70The reason behind this is that currently, netd is stateless. Even though most
71of the networking code can work even in presence of multiple active NIC
72devices of different type, netd is still not completely ready for this
73possibility.
74
75--------------------------------------------
76
77
78