430 lines
15 KiB
TeX
Executable File
430 lines
15 KiB
TeX
Executable File
\documentstyle[12pt,twoside]{article}
|
|
\def\TITLE{IPv6 Flow Labels}
|
|
\input preamble
|
|
\begin{center}
|
|
\Large\bf IPv6 Flow Labels in Linux-2.2.
|
|
\end{center}
|
|
|
|
|
|
\begin{center}
|
|
{ \large Alexey~N.~Kuznetsov } \\
|
|
\em Institute for Nuclear Research, Moscow \\
|
|
\verb|kuznet@ms2.inr.ac.ru| \\
|
|
\rm April 11, 1999
|
|
\end{center}
|
|
|
|
\vspace{5mm}
|
|
|
|
\tableofcontents
|
|
|
|
\section{Introduction.}
|
|
|
|
Every IPv6 packet carries 28 bits of flow information. RFC2460 splits
|
|
these bits to two fields: 8 bits of traffic class (or DS field, if you
|
|
prefer this term) and 20 bits of flow label. Currently there exist
|
|
no well-defined API to manage IPv6 flow information. In this document
|
|
I describe an attempt to design the API for Linux-2.2 IPv6 stack.
|
|
|
|
\vskip 1mm
|
|
|
|
The API must solve the following tasks:
|
|
|
|
\begin{enumerate}
|
|
|
|
\item To allow user to set traffic class bits.
|
|
|
|
\item To allow user to read traffic class bits of received packets.
|
|
This feature is not so useful as the first one, however it will be
|
|
necessary f.e.\ to implement ECN [RFC2481] for datagram oriented services
|
|
or to implement receiver side of SRP or another end-to-end protocol
|
|
using traffic class bits.
|
|
|
|
\item To assign flow labels to packets sent by user.
|
|
|
|
\item To get flow labels of received packets. I do not know
|
|
any applications of this feature, but it is possible that receiver will
|
|
want to use flow labels to distinguish sub-flows.
|
|
|
|
\item To allocate flow labels in the way, compliant to RFC2460. Namely:
|
|
|
|
\begin{itemize}
|
|
\item
|
|
Flow labels must be uniformly distributed (pseudo-)random numbers,
|
|
so that any subset of 20 bits can be used as hash key.
|
|
|
|
\item
|
|
Flows with coinciding source address and flow label must have identical
|
|
destination address and not-fragmentable extensions headers (i.e.\
|
|
hop by hop options and all the headers up to and including routing header,
|
|
if it is present.)
|
|
|
|
\begin{NB}
|
|
There is a hole in specs: some hop-by-hop options can be
|
|
defined only on per-packet base (f.e.\ jumbo payload option).
|
|
Essentially, it means that such options cannot present in packets
|
|
with flow labels.
|
|
\end{NB}
|
|
\begin{NB}
|
|
NB notes here and below reflect only my personal opinion,
|
|
they should be read with smile or should not be read at all :-).
|
|
\end{NB}
|
|
|
|
|
|
\item
|
|
Flow labels have finite lifetime and source is not allowed to reuse
|
|
flow label for another flow within the maximal lifetime has expired,
|
|
so that intermediate nodes will be able to invalidate flow state before
|
|
the label is taken over by another flow.
|
|
Flow state, including lifetime, is propagated along datagram path
|
|
by some application specific methods
|
|
(f.e.\ in RSVP PATH messages or in some hop-by-hop option).
|
|
|
|
|
|
\end{itemize}
|
|
|
|
\end{enumerate}
|
|
|
|
\section{Sending/receiving flow information.}
|
|
|
|
\paragraph{Discussion.}
|
|
\addcontentsline{toc}{subsection}{Discussion}
|
|
It was proposed (Where? I do not remember any explicit statement)
|
|
to solve the first four tasks using
|
|
\verb|sin6_flowinfo| field added to \verb|struct| \verb|sockaddr_in6|
|
|
(see RFC2553).
|
|
|
|
\begin{NB}
|
|
This method is difficult to consider as reasonable, because it
|
|
puts additional overhead to all the services, despite of only
|
|
very small subset of them (none, to be more exact) really use it.
|
|
It contradicts both to IETF spirit and the letter. Before RFC2553
|
|
one justification existed, IPv6 address alignment left 4 byte
|
|
hole in \verb|sockaddr_in6| in any case. Now it has no justification.
|
|
\end{NB}
|
|
|
|
We have two problems with this method. The first one is common for all OSes:
|
|
if \verb|recvmsg()| initializes \verb|sin6_flowinfo| to flow info
|
|
of received packet, we loose one very important property of BSD socket API,
|
|
namely, we are not allowed to use received address for reply directly
|
|
and have to mangle it, even if we are not interested in flowinfo subtleties.
|
|
|
|
\begin{NB}
|
|
RFC2553 adds new requirement: to clear \verb|sin6_flowinfo|.
|
|
Certainly, it is not solution but rather attempt to force applications
|
|
to make unnecessary work. Well, as usually, one mistake in design
|
|
is followed by attempts to patch the hole and more mistakes...
|
|
\end{NB}
|
|
|
|
Another problem is Linux specific. Historically Linux IPv6 did not
|
|
initialize \verb|sin6_flowinfo| at all, so that, if kernel does not
|
|
support flow labels, this field is not zero, but a random number.
|
|
Some applications also did not take care about it.
|
|
|
|
\begin{NB}
|
|
Following RFC2553 such applications can be considered as broken,
|
|
but I still think that they are right: clearing all the address
|
|
before filling known fields is robust but stupid solution.
|
|
Useless wasting CPU cycles and
|
|
memory bandwidth is not a good idea. Such patches are acceptable
|
|
as temporary hacks, but not as standard of the future.
|
|
\end{NB}
|
|
|
|
|
|
\paragraph{Implementation.}
|
|
\addcontentsline{toc}{subsection}{Implementation}
|
|
By default Linux IPv6 does not read \verb|sin6_flowinfo| field
|
|
assuming that common applications are not obliged to initialize it
|
|
and are permitted to consider it as pure alignment padding.
|
|
In order to tell kernel that application
|
|
is aware of this field, it is necessary to set socket option
|
|
\verb|IPV6_FLOWINFO_SEND|.
|
|
|
|
\begin{verbatim}
|
|
int on = 1;
|
|
setsockopt(sock, SOL_IPV6, IPV6_FLOWINFO_SEND,
|
|
(void*)&on, sizeof(on));
|
|
\end{verbatim}
|
|
|
|
Linux kernel never fills \verb|sin6_flowinfo| field, when passing
|
|
message to user space, though the kernels which support flow labels
|
|
initialize it to zero. If user wants to get received flowinfo, he
|
|
will set option \verb|IPV6_FLOWINFO| and after this he will receive
|
|
flowinfo as ancillary data object of type \verb|IPV6_FLOWINFO|
|
|
(cf.\ RFC2292).
|
|
|
|
\begin{verbatim}
|
|
int on = 1;
|
|
setsockopt(sock, SOL_IPV6, IPV6_FLOWINFO, (void*)&on, sizeof(on));
|
|
\end{verbatim}
|
|
|
|
Flowinfo received and latched by a connected TCP socket also may be fetched
|
|
with \verb|getsockopt()| \verb|IPV6_PKTOPTIONS| together with
|
|
another optional information.
|
|
|
|
Besides that, in the spirit of RFC2292 the option \verb|IPV6_FLOWINFO|
|
|
may be used as alternative way to send flowinfo with \verb|sendmsg()| or
|
|
to latch it with \verb|IPV6_PKTOPTIONS|.
|
|
|
|
\paragraph{Note about IPv6 options and destination address.}
|
|
\addcontentsline{toc}{subsection}{IPv6 options and destination address}
|
|
If \verb|sin6_flowinfo| does contain not zero flow label,
|
|
destination address in \verb|sin6_addr| and non-fragmentable
|
|
extension headers are ignored. Instead, kernel uses the values
|
|
cached at flow setup (see below). However, for connected sockets
|
|
kernel prefers the values set at connection time.
|
|
|
|
\paragraph{Example.}
|
|
\addcontentsline{toc}{subsection}{Example}
|
|
After setting socket option \verb|IPV6_FLOWINFO|
|
|
flowlabel and DS field are received as ancillary data object
|
|
of type \verb|IPV6_FLOWINFO| and level \verb|SOL_IPV6|.
|
|
In the cases when it is convenient to use \verb|recvfrom(2)|,
|
|
it is possible to replace library variant with your own one,
|
|
sort of:
|
|
|
|
\begin{verbatim}
|
|
#include <sys/socket.h>
|
|
#include <netinet/in6.h>
|
|
|
|
size_t recvfrom(int fd, char *buf, size_t len, int flags,
|
|
struct sockaddr *addr, int *addrlen)
|
|
{
|
|
size_t cc;
|
|
char cbuf[128];
|
|
struct cmsghdr *c;
|
|
struct iovec iov = { buf, len };
|
|
struct msghdr msg = { addr, *addrlen,
|
|
&iov, 1,
|
|
cbuf, sizeof(cbuf),
|
|
0 };
|
|
|
|
cc = recvmsg(fd, &msg, flags);
|
|
if (cc < 0)
|
|
return cc;
|
|
((struct sockaddr_in6*)addr)->sin6_flowinfo = 0;
|
|
*addrlen = msg.msg_namelen;
|
|
for (c=CMSG_FIRSTHDR(&msg); c; c = CMSG_NEXTHDR(&msg, c)) {
|
|
if (c->cmsg_level != SOL_IPV6 ||
|
|
c->cmsg_type != IPV6_FLOWINFO)
|
|
continue;
|
|
((struct sockaddr_in6*)addr)->sin6_flowinfo = *(__u32*)CMSG_DATA(c);
|
|
}
|
|
return cc;
|
|
}
|
|
\end{verbatim}
|
|
|
|
|
|
|
|
\section{Flow label management.}
|
|
|
|
\paragraph{Discussion.}
|
|
\addcontentsline{toc}{subsection}{Discussion}
|
|
Requirements of RFC2460 are pretty tough. Particularly, lifetimes
|
|
longer than boot time require to store allocated labels at stable
|
|
storage, so that the full implementation necessarily includes user space flow
|
|
label manager. There are at least three different approaches:
|
|
|
|
\begin{enumerate}
|
|
\item {\bf ``Cooperative''. } We could leave flow label allocation wholly
|
|
to user space. When user needs label he requests manager directly. The approach
|
|
is valid, but as any ``cooperative'' approach it suffers of security problems.
|
|
|
|
\begin{NB}
|
|
One idea is to disallow not privileged user to allocate flow
|
|
labels, but instead to pass the socket to manager via \verb|SCM_RIGHTS|
|
|
control message, so that it will allocate label and assign it to socket
|
|
itself. Hmm... the idea is interesting.
|
|
\end{NB}
|
|
|
|
\item {\bf ``Indirect''.} Kernel redirects requests to user level daemon
|
|
and does not install label until the daemon acknowledged the request.
|
|
The approach is the most promising, it is especially pleasant to recognize
|
|
parallel with IPsec API [RFC2367,Craig]. Actually, it may share API with
|
|
IPsec.
|
|
|
|
\item {\bf ``Stupid''.} To allocate labels in kernel space. It is the simplest
|
|
method, but it suffers of two serious flaws: the first,
|
|
we cannot lease labels with lifetimes longer than boot time, the second,
|
|
it is sensitive to DoS attacks. Kernel have to remember all the obsolete
|
|
labels until their expiration and malicious user may fastly eat all the
|
|
flow label space.
|
|
|
|
\end{enumerate}
|
|
|
|
Certainly, I choose the most ``stupid'' method. It is the cheapest one
|
|
for implementor (i.e.\ me), and taking into account that flow labels
|
|
still have no serious applications it is not useful to work on more
|
|
advanced API, especially, taking into account that eventually we
|
|
will get it for no fee together with IPsec.
|
|
|
|
|
|
\paragraph{Implementation.}
|
|
\addcontentsline{toc}{subsection}{Implementation}
|
|
Socket option \verb|IPV6_FLOWLABEL_MGR| allows to
|
|
request flow label manager to allocate new flow label, to reuse
|
|
already allocated one or to delete old flow label.
|
|
Its argument is \verb|struct| \verb|in6_flowlabel_req|:
|
|
|
|
\begin{verbatim}
|
|
struct in6_flowlabel_req
|
|
{
|
|
struct in6_addr flr_dst;
|
|
__u32 flr_label;
|
|
__u8 flr_action;
|
|
__u8 flr_share;
|
|
__u16 flr_flags;
|
|
__u16 flr_expires;
|
|
__u16 flr_linger;
|
|
__u32 __flr_reserved;
|
|
/* Options in format of IPV6_PKTOPTIONS */
|
|
};
|
|
\end{verbatim}
|
|
|
|
\begin{itemize}
|
|
|
|
\item \verb|dst| is IPv6 destination address associated with the label.
|
|
|
|
\item \verb|label| is flow label value in network byte order. If it is zero,
|
|
kernel will allocate new pseudo-random number. Otherwise, kernel will try
|
|
to lease flow label ordered by user. In this case, it is user task to provide
|
|
necessary flow label randomness.
|
|
|
|
\item \verb|action| is requested operation. Currently, only three operations
|
|
are defined:
|
|
|
|
\begin{verbatim}
|
|
#define IPV6_FL_A_GET 0 /* Get flow label */
|
|
#define IPV6_FL_A_PUT 1 /* Release flow label */
|
|
#define IPV6_FL_A_RENEW 2 /* Update expire time */
|
|
\end{verbatim}
|
|
|
|
\item \verb|flags| are optional modifiers. Currently
|
|
only \verb|IPV6_FL_A_GET| has modifiers:
|
|
|
|
\begin{verbatim}
|
|
#define IPV6_FL_F_CREATE 1 /* Allowed to create new label */
|
|
#define IPV6_FL_F_EXCL 2 /* Do not create new label */
|
|
\end{verbatim}
|
|
|
|
|
|
\item \verb|share| defines who is allowed to reuse the same flow label.
|
|
|
|
\begin{verbatim}
|
|
#define IPV6_FL_S_NONE 0 /* Not defined */
|
|
#define IPV6_FL_S_EXCL 1 /* Label is private */
|
|
#define IPV6_FL_S_PROCESS 2 /* May be reused by this process */
|
|
#define IPV6_FL_S_USER 3 /* May be reused by this user */
|
|
#define IPV6_FL_S_ANY 255 /* Anyone may reuse it */
|
|
\end{verbatim}
|
|
|
|
\item \verb|linger| is time in seconds. After the last user releases flow
|
|
label, it will not be reused with different destination and options at least
|
|
during this time. If \verb|share| is not \verb|IPV6_FL_S_EXCL| the label
|
|
still can be shared by another sockets. Current implementation does not allow
|
|
unprivileged user to set linger longer than 60 sec.
|
|
|
|
\item \verb|expires| is time in seconds. Flow label will be kept at least
|
|
for this time, but it will not be destroyed before user released it explicitly
|
|
or closed all the sockets using it. Current implementation does not allow
|
|
unprivileged user to set timeout longer than 60 sec. Proviledged applications
|
|
MAY set longer lifetimes, but in this case they MUST save allocated
|
|
labels at stable storage and restore them back after reboot before the first
|
|
application allocates new flow.
|
|
|
|
\end{itemize}
|
|
|
|
This structure is followed by optional extension headers associated
|
|
with this flow label in format of \verb|IPV6_PKTOPTIONS|. Only
|
|
\verb|IPV6_HOPOPTS|, \verb|IPV6_RTHDR| and, if \verb|IPV6_RTHDR| presents,
|
|
\verb|IPV6_DSTOPTS| are allowed.
|
|
|
|
\paragraph{Example.}
|
|
\addcontentsline{toc}{subsection}{Example}
|
|
The function \verb|get_flow_label| allocates
|
|
private flow label.
|
|
|
|
\begin{verbatim}
|
|
int get_flow_label(int fd, struct sockaddr_in6 *dst, __u32 fl)
|
|
{
|
|
int on = 1;
|
|
struct in6_flowlabel_req freq;
|
|
|
|
memset(&freq, 0, sizeof(freq));
|
|
freq.flr_label = htonl(fl);
|
|
freq.flr_action = IPV6_FL_A_GET;
|
|
freq.flr_flags = IPV6_FL_F_CREATE | IPV6_FL_F_EXCL;
|
|
freq.flr_share = IPV6_FL_S_EXCL;
|
|
memcpy(&freq.flr_dst, &dst->sin6_addr, 16);
|
|
if (setsockopt(fd, SOL_IPV6, IPV6_FLOWLABEL_MGR,
|
|
&freq, sizeof(freq)) == -1) {
|
|
perror ("can't lease flowlabel");
|
|
return -1;
|
|
}
|
|
dst->sin6_flowinfo |= freq.flr_label;
|
|
|
|
if (setsockopt(fd, SOL_IPV6, IPV6_FLOWINFO_SEND,
|
|
&on, sizeof(on)) == -1) {
|
|
perror ("can't send flowinfo");
|
|
|
|
freq.flr_action = IPV6_FL_A_PUT;
|
|
setsockopt(fd, SOL_IPV6, IPV6_FLOWLABEL_MGR,
|
|
&freq, sizeof(freq));
|
|
return -1;
|
|
}
|
|
return 0;
|
|
}
|
|
\end{verbatim}
|
|
|
|
A bit more complicated example using routing header can be found
|
|
in \verb|ping6| utility (\verb|iputils| package). Linux rsvpd backend
|
|
contains an example of using operation \verb|IPV6_FL_A_RENEW|.
|
|
|
|
\paragraph{Listing flow labels.}
|
|
\addcontentsline{toc}{subsection}{Listing flow labels}
|
|
List of currently allocated
|
|
flow labels may be read from \verb|/proc/net/ip6_flowlabel|.
|
|
|
|
\begin{verbatim}
|
|
Label S Owner Users Linger Expires Dst Opt
|
|
A1BE5 1 0 0 6 3 3ffe2400000000010a0020fffe71fb30 0
|
|
\end{verbatim}
|
|
|
|
\begin{itemize}
|
|
\item \verb|Label| is hexadecimal flow label value.
|
|
\item \verb|S| is sharing style.
|
|
\item \verb|Owner| is ID of creator, it is zero, pid or uid, depending on
|
|
sharing style.
|
|
\item \verb|Users| is number of applications using the label now.
|
|
\item \verb|Linger| is \verb|linger| of this label in seconds.
|
|
\item \verb|Expires| is time until expiration of the label in seconds. It may
|
|
be negative, if the label is in use.
|
|
\item \verb|Dst| is IPv6 destination address.
|
|
\item \verb|Opt| is length of options, associated with the label. Option
|
|
data are not accessible.
|
|
\end{itemize}
|
|
|
|
|
|
\paragraph{Flow labels and RSVP.}
|
|
\addcontentsline{toc}{subsection}{Flow labels and RSVP}
|
|
RSVP daemon supports IPv6 flow labels
|
|
without any modifications to standard ISI RAPI. Sender must allocate
|
|
flow label, fill corresponding sender template and submit it to local rsvp
|
|
daemon. rsvpd will check the label and start to announce it in PATH
|
|
messages. Rsvpd on sender node will renew the flow label, so that it will not
|
|
be reused before path state expires and all the intermediate
|
|
routers and receiver purge flow state.
|
|
|
|
\verb|rtap| utility is modified to parse flow labels. F.e.\ if user allocated
|
|
flow label \verb|0xA1234|, he may write:
|
|
|
|
\begin{verbatim}
|
|
RTAP> sender 3ffe:2400::1/FL0xA1234 <Tspec>
|
|
\end{verbatim}
|
|
|
|
Receiver makes reservation with command:
|
|
\begin{verbatim}
|
|
RTAP> reserve ff 3ffe:2400::1/FL0xA1234 <Flowspec>
|
|
\end{verbatim}
|
|
|
|
\end{document}
|