Skip to content

GitLab

Commit c50d6f5b authored by Steven Allen's avatar Steven Allen Committed by GitHub
Browse files

Merge pull request #216 from libp2p/feat/summer-cleaning 

Summer cleaning
parents 4aa941a4 3d33ba19
Hide whitespace changes
Inline Side-by-side
Showing
with 179 additions and 146 deletions
+179 -146
CONTRIBUTE.md deleted 100644 → 0
View file @ 4aa941a4
# Contribute
go-libp2p is part of [The IPFS Project](https://github.com/ipfs/ipfs), and is MIT licensed open source software. We welcome contributions big and small! Take a look at the [community contributing notes](https://github.com/ipfs/community/blob/master/contributing.md). Please make sure to check the [issues](https://github.com/ipfs/go-libp2p/issues). Search the closed ones
before reporting things, and help us with the open ones.
Guidelines:
- read the [libp2p spec](https://github.com/libp2p/specs)
- please make branches + pull-request, even if working on the main repository
- ask questions or talk about things in [Issues](https://github.com/ipfs/go-ipfs/issues) or #ipfs on freenode.
- ensure you are able to contribute (no legal issues please-- we use the DCO)
- run `go fmt` before pushing any code
- run `golint` and `go vet` too -- some things (like protobuf files) are expected to fail.
- get in touch with @jbenet and @diasdavid about how best to contribute
- have fun!
Makefile
View file @ c50d6f5b
......@@ -2,7 +2,7 @@ gx:
go get github.com/whyrusleeping/gx
go get github.com/whyrusleeping/gx-go
deps-examples: deps
deps-protocol-muxing: deps
go get -u github.com/multiformats/go-multicodec
go get -u github.com/jbenet/go-msgio
......@@ -12,4 +12,3 @@ deps: gx
publish:
gx-go rewrite --undo
README.md
View file @ c50d6f5b
[libp2p](https://github.com/libp2p/specs) implementation in Go.
===================
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![GoDoc](https://godoc.org/github.com/ipfs/go-libp2p?status.svg)](https://godoc.org/github.com/libp2p/go-libp2p)
[![Build Status](https://travis-ci.org/ipfs/go-libp2p.svg?branch=master)](https://travis-ci.org/libp2p/go-libp2p)
![](https://raw.githubusercontent.com/diasdavid/specs/libp2p-spec/protocol/network/figs/logo.png)
> libp2p implementation in Go
# Description
<h1 align="center">
<a href="libp2p.io"><img width="250" src="https://github.com/libp2p/libp2p/blob/master/logo/alternates/libp2p-logo-alt-2.png?raw=true" alt="libp2p hex logo" /></a>
</h1>
<h3 align="center">The Go implementation of the libp2p Networking Stack.</h3>
<p align="center">
<a href="http://ipn.io"><img src="https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square" /></a>
<a href="http://libp2p.io/"><img src="https://img.shields.io/badge/project-libp2p-blue.svg?style=flat-square" /></a>
<a href="http://webchat.freenode.net/?channels=%23ipfs"><img src="https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square" /></a>
<a href="https://waffle.io/libp2p/libp2p"><img src="https://img.shields.io/badge/pm-waffle-blue.svg?style=flat-square" /></a>
</p>
<p align="center">
<a href="https://travis-ci.org/libp2p/go-libp2p"><img src="https://travis-ci.org/libp2p/go-libp2p.svg?branch=master" /></a>
<!--<a href="https://circleci.com/gh/libp2p/go-libp2p"><img src="https://circleci.com/gh/libp2p/go-libp2p.svg?style=svg" /></a>-->
<!--<a href="https://coveralls.io/github/libp2p/go-libp2p?branch=master"><img src="https://coveralls.io/repos/github/libp2p/go-libp2p/badge.svg?branch=master"></a>-->
<br>
<a href="https://github.com/RichardLitt/standard-readme"><img src="https://img.shields.io/badge/standard--readme-OK-green.svg?style=flat-square" /></a>
<a href="https://godoc.org/github.com/libp2p/go-libp2p"><img src="https://godoc.org/github.com/ipfs/go-libp2p?status.svg" /></a>
<a href=""><img src="https://img.shields.io/badge/golang-%3E%3D1.8.0-orange.svg?style=flat-square" /></a>
<br>
</p>
# Project status
[![Throughput Graph](https://graphs.waffle.io/libp2p/go-libp2p/throughput.svg)](https://waffle.io/libp2p/go-libp2p/metrics/throughput)
# Table of Contents
- [Background](#background)
- [Bundles](#bundles)
- [Usage](#usage)
- [Install](#install)
- [API](#api)
- [Examples](#examples)
- [Development](#development)
- [Tests](#tests)
- [Packages](#packages)
- [Contribute](#contribute)
- [License](#license)
## Background
[libp2p](https://github.com/libp2p/specs) is a networking stack and library modularized out of [The IPFS Project](https://github.com/ipfs/ipfs), and bundled separately for other tools to use.
>
......@@ -18,30 +48,33 @@ libp2p is the product of a long, and arduous quest of understanding -- a deep di
>
> We will be writing a set of docs, posts, tutorials, and talks to explain what p2p is, why it is tremendously useful, and how it can help your existing and new projects. But in the meantime, check out
>
> - [**The IPFS Network Spec**](https://github.com/libp2p/specs), which grew into libp2p
> - [**The libp2p Specification**](https://github.com/libp2p/specs)
> - [**go-libp2p implementation**](https://github.com/libp2p/go-libp2p)
> - [**js-libp2p implementation**](https://github.com/libp2p/js-libp2p)
# Contribute
libp2p implementation in Go is a work in progress. As such, there's a few things you can do right now to help out:
- Go through the modules below and **check out existing issues**. This would be especially useful for modules in active development. Some knowledge of IPFS/libp2p may be required, as well as the infrasture behind it - for instance, you may need to read up on p2p and more complex operations like muxing to be able to help technically.
- **Perform code reviews**.
- **Add tests**. There can never be enough tests.
## Bundles
There is currently only one bundle of `go-libp2p`, this package. This bundle is used by [`go-ipfs`](https://github.com/ipfs/go-ipfs).
# Usage
## Usage
`go-libp2p` repo will be a place holder for the list of Go modules that compose Go libp2p, as well as its entry point.
`go-libp2p` repo is a place holder for the list of Go modules that compose Go libp2p, as well as its entry point.
## Install
### Install
```bash
$ go get -d github.com/libp2p/go-libp2p
$ cd $GOPATH/src/github.com/libp2p/go-libp2p
$ make
> go get -d github.com/libp2p/go-libp2p
> cd $GOPATH/src/github.com/libp2p/go-libp2p
> make
> make deps
```
# Examples
### API
[![GoDoc](https://godoc.org/github.com/ipfs/go-libp2p?status.svg)](https://godoc.org/github.com/libp2p/go-libp2p)
### Examples
Examples can be found on the [examples folder](examples).
......@@ -50,18 +83,72 @@ Examples can be found on the [examples folder](examples).
```bash
$ cd $GOPATH/src/github.com/libp2p/go-libp2p
$ make deps
$ go test ./p2p/<path of module you want to run tests for>
$ gx test ./p2p/<path of module you want to run tests for>
```
## Development
### Dependencies
While developing, you need to use [gx to install and link your dependencies](https://github.com/whyrusleeping/gx#dependencies), to do that, run:
```sh
> make deps
```
## Links
- [**Specs**](https://github.com/libp2p/specs)
- [**Website**](https://github.com/libp2p/website)
Before commiting and pushing to Github, make sure to rewind the gx'ify of dependencies. You can do that with:
```sh
> make publish
```
### Tests
Running of individual tests is done through `gx test <path to test>`
### Packages
> **WIP**
List of packages currently in existence for libp2p:
| Package | Version | CI |
|--------------------|---------|---------------------|
| **Transports** |
| **Connection Upgrades** |
| **Stream Muxers** |
| **Discovery** |
| **Crypto Channels** |
| **Peer Routing** |
| **Content Routing** |
| **Miscellaneous** |
| **Data Types** |
# Contribute
go-libp2p is part of [The IPFS Project](https://github.com/ipfs/ipfs), and is MIT licensed open source software. We welcome contributions big and small! Take a look at the [community contributing notes](https://github.com/ipfs/community/blob/master/contributing.md). Please make sure to check the [issues](https://github.com/ipfs/go-libp2p/issues). Search the closed ones before reporting things, and help us with the open ones.
Guidelines:
- read the [libp2p spec](https://github.com/libp2p/specs)
- please make branches + pull-request, even if working on the main repository
- ask questions or talk about things in [Issues](https://github.com/libp2p/go-libp2p/issues) or #ipfs on freenode.
- ensure you are able to contribute (no legal issues please-- we use the DCO)
- run `go fmt` before pushing any code
- run `golint` and `go vet` too -- some things (like protobuf files) are expected to fail.
- get in touch with @jbenet and @diasdavid about how best to contribute
- have fun!
There's a few things you can do right now to help out:
- Go through the modules below and **check out existing issues**. This would be especially useful for modules in active development. Some knowledge of IPFS/libp2p may be required, as well as the infrasture behind it - for instance, you may need to read up on p2p and more complex operations like muxing to be able to help technically.
- **Perform code reviews**.
- **Add tests**. There can never be enough tests.
## Modularizing go-libp2p
## Extracting packages from go-libp2p
We have currently a work in progress of modularizing go-libp2p from a repo monolith to several packages in different repos that can be reused for other projects of swapped for custom libp2p builds.
We want to maintain history, so we'll use git-subtree for extracting packages.
We want to maintain history, so we'll use git-subtree for extracting packages. Find instructions below:
```sh
# 1) create the extracted tree (has the directory specified as -P as its root)
......
TODO deleted 100644 → 0
View file @ 4aa941a4
- pull mdns out as its own separate package
examples/README.md 0 → 100644
View file @ c50d6f5b
# `go-libp2p` examples and tutorials
In this folder, you can find a variety of examples to help you get started in using go-libp2p. Every example as a specific purpose and some of each incorporate a full tutorial that you can follow through, helping you expand your knowledge about libp2p and p2p networks in general.
Let us know if you find any issue or if you want to contribute and add a new tutorial, feel welcome to submit a pr, thank you!
## Examples and Tutorials
- [The libp2p 'host'](./libp2p-host)
- [Building an http proxy with libp2p](./http-proxy)
- [Protocol Multiplexing with multicodecs](./protocol-multiplexing-with-multicodecs)
- [An echo host](./echo)
examples/echo/README.md
View file @ c50d6f5b
# Echo client/server with libp2p
This is an example that quickly shows how to use the `go-libp2p` stack,
including Host/Basichost, Network/Swarm, Streams, Peerstores and
Multiaddresses.
This is an example that quickly shows how to use the `go-libp2p` stack, including Host/Basichost, Network/Swarm, Streams, Peerstores and Multiaddresses.
This example can be started in either listen mode, or dial mode.
In listen mode, it will sit and wait for incoming connections on the
`/echo/1.0.0` protocol. Whenever it receives a stream, it will write the
message `"Hello, world!"` over the stream and close it.
In listen mode, it will sit and wait for incoming connections on the `/echo/1.0.0` protocol. Whenever it receives a stream, it will write the message `"Hello, world!"` over the stream and close it.
In dial mode, the node will start up, connect to the given address, open a
stream to the target peer, and read a message on the protocol `/echo/1.0.0`.
In dial mode, the node will start up, connect to the given address, open a stream to the target peer, and read a message on the protocol `/echo/1.0.0`.
## Build
......@@ -31,8 +26,7 @@ From `go-libp2p` base folder:
2017/03/15 14:11:32 listening for connections
```
The listener libp2p host will print its `Multiaddress`, which indicates how it
can be reached (ip4+tcp) and its randomly generated ID (`QmYo41Gyb...`)
The listener libp2p host will print its `Multiaddress`, which indicates how it can be reached (ip4+tcp) and its randomly generated ID (`QmYo41Gyb...`)
Now, launch another node that talks to the listener:
......@@ -40,52 +34,19 @@ Now, launch another node that talks to the listener:
> ./echo -l 10001 -d /ip4/127.0.0.1/tcp/10000/ipfs/QmYo41GybvrXk8y8Xnm1P7pfA4YEXCpfnLyzgRPnNbG35e
```
The new node with send the message `"Hello, world!"` to the
listener, which will in turn echo it over the stream and close it. The
listener logs the message, and the sender logs the response.
The new node with send the message `"Hello, world!"` to the listener, which will in turn echo it over the stream and close it. The listener logs the message, and the sender logs the response.
## Details
The `makeBasicHost()` function creates a
[go-libp2p-basichost](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic)
object. `basichost` objects wrap
[go-libp2 swarms](https://godoc.org/github.com/libp2p/go-libp2p-swarm#Swarm)
and should be used preferentially. A
[go-libp2p-swarm Network](https://godoc.org/github.com/libp2p/go-libp2p-swarm#Network)
is a `swarm` which complies to the
[go-libp2p-net Network interface](https://godoc.org/github.com/libp2p/go-libp2p-net#Network)
and takes care of maintaining streams, connections, multiplexing different
protocols on them, handling incoming connections etc.
The `makeBasicHost()` function creates a [go-libp2p-basichost](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic) object. `basichost` objects wrap [go-libp2 swarms](https://godoc.org/github.com/libp2p/go-libp2p-swarm#Swarm) and should be used preferentially. A [go-libp2p-swarm Network](https://godoc.org/github.com/libp2p/go-libp2p-swarm#Network) is a `swarm` which complies to the [go-libp2p-net Network interface](https://godoc.org/github.com/libp2p/go-libp2p-net#Network) and takes care of maintaining streams, connections, multiplexing different protocols on them, handling incoming connections etc.
In order to create the swarm (and a `basichost`), the example needs:
* An
[ipfs-procotol ID](https://godoc.org/github.com/libp2p/go-libp2p-peer#ID)
like `QmNtX1cvrm2K6mQmMEaMxAuB4rTexhd87vpYVot4sEZzxc`. The example
autogenerates a key pair on every run and uses an ID extracted from the
public key (the hash of the public key). When using `-secio`, it uses
the key pair to encrypt communications.
* A [Multiaddress](https://godoc.org/github.com/multiformats/go-multiaddr),
which indicates how to reach this peer. There can be several of them
(using different protocols or locations for example). Example:
`/ip4/127.0.0.1/tcp/1234`.
* A
[go-libp2p-peerstore](https://godoc.org/github.com/libp2p/go-libp2p-peerstore),
which is used as a address book which matches node IDs to the
multiaddresses through which they can be contacted. This peerstore gets
autopopulated when manually opening a connection (with
[`Connect()`](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.Connect). Alternatively,
we can manually
[`AddAddr()`](https://godoc.org/github.com/libp2p/go-libp2p-peerstore#AddrManager.AddAddr)
as in the example.
A `basichost` can now open streams (bi-directional channel between to peers)
using
[NewStream](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.NewStream)
and use them to send and receive data tagged with a `Protocol.ID` (a
string). The host can also listen for incoming connections for a given
`Protocol` with
[`SetStreamHandle()`](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.SetStreamHandler).
The example makes use of all of this to enable communication between a
listener and a sender using protocol `/echo/1.0.0` (which could be any other thing).
- An [ipfs-procotol ID](https://godoc.org/github.com/libp2p/go-libp2p-peer#ID) like `QmNtX1cvrm2K6mQmMEaMxAuB4rTexhd87vpYVot4sEZzxc`. The example autogenerates a key pair on every run and uses an ID extracted from the public key (the hash of the public key). When using `-secio`, it uses the key pair to encrypt communications.
- A [Multiaddress](https://godoc.org/github.com/multiformats/go-multiaddr), which indicates how to reach this peer. There can be several of them (using different protocols or locations for example). Example: `/ip4/127.0.0.1/tcp/1234`.
- A [go-libp2p-peerstore](https://godoc.org/github.com/libp2p/go-libp2p-peerstore), which is used as a address book which matches node IDs to the multiaddresses through which they can be contacted. This peerstore gets autopopulated when manually opening a connection (with [`Connect()`](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.Connect). Alternatively, we can manually [`AddAddr()`](https://godoc.org/github.com/libp2p/go-libp2p-peerstore#AddrManager.AddAddr) as in the example.
A `basichost` can now open streams (bi-directional channel between to peers) using [NewStream](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.NewStream) and use them to send and receive data tagged with a `Protocol.ID` (a string). The host can also listen for incoming connections for a given
`Protocol` with [`SetStreamHandle()`](https://godoc.org/github.com/libp2p/go-libp2p/p2p/host/basic#BasicHost.SetStreamHandler).
The example makes use of all of this to enable communication between a listener and a sender using protocol `/echo/1.0.0` (which could be any other thing).
examples/http-proxy/README.md
View file @ c50d6f5b
......@@ -2,24 +2,21 @@
This examples shows how to create a simple HTTP proxy service with libp2p:
```
XXX
XX XXXXXX
X XX
XXXXXXX XX XX XXXXXXXXXX
+---------------------+ +---------------------+ XXX XXX XXX XXX
HTTP Request | | | | XX XX
+-------------------> | libp2p stream | | HTTP X X
| Local peer <------------------> Remote peer <-------------> HTTP SERVER - THE INTERNET XX
<-------------------+ | | | Req & Resp XX X
HTTP Response | libp2p host | | libp2p host | XXXX XXXX XXXXXXXXXXXXXXXXXXXX XXXX
+---------------------+ +---------------------+ XXXXX
XXX
XX XXXXXX
X XX
XXXXXXX XX XX XXXXXXXXXX
+----------------+ +-----------------+ XXX XXX XXX XXX
HTTP Request | | | | XX XX
+-----------------> | libp2p stream | | HTTP X X
| Local peer <----------------> Remote peer <-------------> HTTP SERVER - THE INTERNET XX
<-----------------+ | | | Req & Resp XX X
HTTP Response | libp2p host | | libp2p host | XXXX XXXX XXXXXXXXXXXXXXXXXXXX XXXX
+----------------+ +-----------------+ XXXXX
```
In order to proxy an HTTP request, we create a local peer which listens on `localhost:9900`. HTTP requests performed to that address are tunneled via a libp2p stream to a remote peer, which then performs the HTTP requests and sends the response back to the local peer, which relays it
to the user.
In order to proxy an HTTP request, we create a local peer which listens on `localhost:9900`. HTTP requests performed to that address are tunneled via a libp2p stream to a remote peer, which then performs the HTTP requests and sends the response back to the local peer, which relays it to the user.
Note that this is a very simple approach to a proxy, and does not perform any header management, nor supports HTTPS. The `proxy.go` code is thoroughly commeted, detailing what is happening in every step.
......@@ -37,7 +34,7 @@ From `go-libp2p` base folder:
First run the "remote" peer as follows. It will print a local peer address. If you would like to run this on a separate machine, please replace the IP accordingly:
```sh
$ ./http-proxy
> ./http-proxy
Proxy server is ready
libp2p-peer addresses:
/ip4/127.0.0.1/tcp/12000/ipfs/QmddTrQXhA9AkCpXPTkcY7e22NK73TwkUms3a44DhTKJTD
......@@ -46,7 +43,7 @@ libp2p-peer addresses:
The run the local peer, indicating that it will need to forward http requests to the remote peer as follows:
```
$ ./http-proxy -d /ip4/127.0.0.1/tcp/12000/ipfs/QmddTrQXhA9AkCpXPTkcY7e22NK73TwkUms3a44DhTKJTD
> ./http-proxy -d /ip4/127.0.0.1/tcp/12000/ipfs/QmddTrQXhA9AkCpXPTkcY7e22NK73TwkUms3a44DhTKJTD
Proxy server is ready
libp2p-peer addresses:
/ip4/127.0.0.1/tcp/12001/ipfs/Qmaa2AYTha1UqcFVX97p9R1UP7vbzDLY7bqWsZw1135QvN
......@@ -56,7 +53,6 @@ proxy listening on 127.0.0.1:9900
As you can see, the proxy prints the listening address `127.0.0.1:9900`. You can now use this address as proxy, for example with `curl`:
```
$ curl -x "127.0.0.1:9900" "http://ipfs.io/ipfs/QmfUX75pGRBRDnjeoMkQzuQczuCup2aYbeLxz5NzeSu9G6"
> curl -x "127.0.0.1:9900" "http://ipfs.io/ipfs/QmfUX75pGRBRDnjeoMkQzuQczuCup2aYbeLxz5NzeSu9G6"
it works!
```
tutorial/001-the-host/README.md → examples/libp2p-host/README.md
View file @ c50d6f5b
# The libp2p 'host'
For most applications, the host is the basic building block you'll need to get
started. This guide will show how to construct and use a simple host.
The host is an abstraction that manages services on top of a swarm. It provides
a clean interface to connect to a service on a given remote peer.
First, you'll need an ID, and a place to store that ID. To generate an
ID, you can do the following:
For most applications, the host is the basic building block you'll need to get started. This guide will show how to construct and use a simple host.
The host is an abstraction that manages services on top of a swarm. It provides a clean interface to connect to a service on a given remote peer.
First, you'll need an ID, and a place to store that ID. To generate an ID, you can do the following:
```go
import (
"crypto/rand"
......@@ -34,8 +34,8 @@ ps.AddPrivKey(pid, priv)
ps.AddPubKey(pid, priv)
```
Next, you'll need at least one address that you want to listen on. You can go
from a string to a multiaddr like this:
Next, you'll need at least one address that you want to listen on. You can go from a string to a multiaddr like this:
```go
import ma "github.com/multiformats/go-multiaddr"
......@@ -47,10 +47,8 @@ if err != nil {
}
```
Now you know who you are, and where you live (in a manner of speaking). The
next step is setting up a 'swarm network' to handle all the peers you will
connect to. The swarm handles incoming connections from other peers, and
handles the negotiation of new outbound connections.
Now you know who you are, and where you live (in a manner of speaking). The next step is setting up a 'swarm network' to handle all the peers you will connect to. The swarm handles incoming connections from other peers, and handles the negotiation of new outbound connections.
```go
import (
"context"
......@@ -67,20 +65,16 @@ if err != nil {
}
```
At this point, we have everything needed to finally construct a host. That call
is the simplest one so far:
At this point, we have everything needed to finally construct a host. That call is the simplest one so far:
```go
import bhost "github.com/libp2p/go-libp2p/p2p/host/basic"
myhost := bhost.New(netw)
```
And thats it, you have a libp2p host and you're ready to start doing some
awesome p2p networking!
And thats it, you have a libp2p host and you're ready to start doing some awesome p2p networking!
In future guides we will go over ways to use hosts, configure them differently
(hint: there are a huge number of ways to set these up), and interesting ways
to apply this technology to various applications you might want to build.
In future guides we will go over ways to use hosts, configure them differently (hint: there are a huge number of ways to set these up), and interesting ways to apply this technology to various applications you might want to build.
To see this code all put together, take a look at the `host.go` file in this
directory.
To see this code all put together, take a look at the `host.go` file in this directory.
examples/multicodecs/README.md → examples/protocol-multiplexing-with-multicodecs/README.md
View file @ c50d6f5b
# Using multicodecs with LibP2P
# Protocol Multiplexing using multicodecs with libp2p
This examples shows how to use multicodecs (i.e. json) to encode and transmit information between LibP2P hosts using LibP2P Streams.
......@@ -11,8 +11,8 @@ This example expects that you area already familiar with the [echo example](http
From `go-libp2p` base folder:
```
> make deps-examples
> go build ./examples/multicodecs
> make deps-protocol-muxing
> go build ./examples/protocol-multiplexing-with-multicodecs
```
## Usage
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment