Skip to content

First iteration of the HTTP client for cpp-netlib v1.0.0a #350

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 105 commits into from
Dec 18, 2013
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
faade8a
Cleaned up thread_pool so that I can read it more clearly.
glynos Jun 23, 2013
19a8b0f
Started HTTP client rewriting by removing some redundant files.
glynos Jun 23, 2013
2b2c2d3
Removed a redundant header from the message framework.
glynos Jun 23, 2013
e1bbd36
Removed basic_storage_* types that were not being used.
glynos Jun 23, 2013
e888689
Removed some more redundant files.
glynos Jun 23, 2013
f169c24
Removed some more files that were not being used.
glynos Jun 23, 2013
297a773
Removed redundant tests.
glynos Jun 23, 2013
8bcce7a
First commit of network::http::v2::client.
glynos Jun 12, 2013
fb7eecf
Updated first set of HTTP client tests.
glynos Jun 19, 2013
6b51a82
Updated client_options tests.
glynos Jun 20, 2013
07e1e8e
Moved some of the tests around; added class that will represent HTTP …
glynos Jun 20, 2013
4eac522
Added a some extra options for the client and the request; removed so…
glynos Jun 23, 2013
71757f1
Updated client sources, adding some more tests.
glynos Jul 1, 2013
73b0831
Added some additional tests for the HTTP URL.
glynos Jul 27, 2013
70a99a9
Added a message_base class.
glynos Jul 27, 2013
40e3b59
Added HTTP constants for client 1.0.x.
glynos Jul 27, 2013
f23c827
Updated and added HTTP constants.
glynos Jul 27, 2013
72ea85e
Updated constants and message headers.
glynos Jul 30, 2013
06cd005
Updated build scripts so as not to build the old client and server; a…
glynos Aug 27, 2013
5ecb5c7
Next iteration of improvements, still a long way from a working HTTP …
glynos Aug 28, 2013
41f4ec2
Added a client_errors file and invalid_scheme class.
glynos Aug 28, 2013
b700bcc
Added tests and methods to the HTTP request.
glynos Aug 30, 2013
6540a84
Added async_resolver.
glynos Aug 31, 2013
7f9f50e
Added logging again to project.
glynos Sep 1, 2013
d653b4b
Added connection_timeout exception.
glynos Sep 1, 2013
055e513
Added resolver_error.
glynos Sep 1, 2013
06b8b94
Added resolver delegate.
glynos Sep 1, 2013
5b82479
Added connection_delegate and test files (no test cases yet).
glynos Sep 1, 2013
88f6faf
Removed message_base.hpp
glynos Sep 1, 2013
0610d08
Updated errors and the HTTP request.
glynos Sep 1, 2013
d9174e6
Updated method type.
glynos Sep 2, 2013
5e68836
Updated requests test to check the port number.
glynos Sep 2, 2013
83cca78
A few updates to the client - next step is to test the components ind…
glynos Sep 5, 2013
b0cdd9a
Added igloo BDD-style feature testing framework.
glynos Sep 5, 2013
7bcf001
Moved the unit tests around.
glynos Sep 5, 2013
b895062
Async resolver tests now use igloo.
glynos Sep 5, 2013
85830b0
Updated feature tests for client and connections.
glynos Sep 6, 2013
dc2d4cb
Updated some stuff that's not important.
glynos Sep 6, 2013
3faaa69
Updated Doxygen comments on almost all files. Added a response_parser…
glynos Sep 7, 2013
3442d76
Resolver now uses asynchronous futures.
glynos Sep 8, 2013
b80ab15
Little hack to make async connection test pass (it's actually synchro…
glynos Sep 8, 2013
22df4eb
Done a lot of stuff to play around with HTTP - it successfully makes …
glynos Sep 8, 2013
7415d62
Updated the resolver to avoid using futures,as this turned to be very…
glynos Sep 10, 2013
8295fb8
Well these test all pass...
glynos Sep 10, 2013
8976b43
Updated response parser tests.
glynos Sep 11, 2013
ebd7567
Updated tests for the response parser, added cancellation member func…
glynos Sep 12, 2013
6a7205e
Mostly updates to documentation.
glynos Sep 21, 2013
3d0cc92
Some more, incomplete documentation.
glynos Sep 23, 2013
af77702
Updates to some of the tests.
glynos Sep 23, 2013
40a2309
Renamed async_resolver_delegate, and removed some of the framework cr…
glynos Oct 4, 2013
bcd5554
Renamed connection_delegate.
glynos Oct 4, 2013
00654de
Removed some unnecessary files.
glynos Oct 6, 2013
3382000
Replaced std::function handler with a template in async_resolver.
glynos Oct 6, 2013
41a1e67
Removed warnings from CMakeLists.txt files, because of the noise from…
glynos Oct 24, 2013
d100aaa
Removed date_time from Boost_COMPONENTS because it is header-only in …
glynos Oct 25, 2013
8847a53
Renamed tests.
glynos Oct 27, 2013
4b1fd17
Rewrote constants.
glynos Oct 27, 2013
ae2c6d8
Updated uri submodule.
glynos Oct 27, 2013
84a555b
Reworked status and HTTP client request.
glynos Oct 27, 2013
603547c
Modification to connection test.
glynos Nov 15, 2013
58a7782
Added an HTTP response error.
glynos Nov 15, 2013
9345bed
Updated tests for the response_parser.
glynos Nov 15, 2013
bb006d6
Added free swap functions to client_options, request_options, request…
glynos Nov 15, 2013
b7e7aa5
Updated the connection and the connection tests.
glynos Nov 15, 2013
28ffad1
Minor refactoring in request, response, client.
glynos Nov 24, 2013
609ed29
Used macro NETWORK_NOEXCEPT instead of keyword noexcept.
glynos Nov 24, 2013
bb42b4f
Fixed include guards.
glynos Nov 24, 2013
7a3e856
Updated uri submodule.
glynos Nov 24, 2013
d945654
Minor changes to the thread_pool class and headers.
glynos Nov 27, 2013
e8d9262
Reverted changes where I used a macro for a noexcept replacement - we…
glynos Nov 27, 2013
74d99b8
Merge remote-tracking branch 'upstream/master' into client-v2
glynos Nov 30, 2013
0c85833
Updated tests for client.
glynos Dec 1, 2013
0739a4a
Added some new HTTP status codes.
glynos Dec 1, 2013
184d278
Updated some HTTP client errors.
glynos Dec 2, 2013
d68fb57
Added a small project for generic network errors.
glynos Dec 2, 2013
d043072
Added doxygen documentation for error handling.
glynos Dec 2, 2013
b469278
Renamed concurrency library.
glynos Dec 2, 2013
4be2b5a
Commented out stuff so the HTTP client test did not crash, at least.
glynos Dec 2, 2013
7afed49
Updated README documentation.
glynos Dec 2, 2013
695e074
Updated the Doxyfile.
glynos Dec 7, 2013
4373fc1
Updated error handling for general networking errors. Added a network…
glynos Dec 7, 2013
b02948f
Replaced the Handler template with std::function.
glynos Dec 7, 2013
29ec7e2
Renamed base class connection to async_connection.
glynos Dec 7, 2013
416fe04
Added an endpoint_cache class; refactored async_resolver.
glynos Dec 7, 2013
58d5f94
Made async_resolver and ABC; added implementation in tcp_resolver.
glynos Dec 7, 2013
44b9185
Test fails with an exception instead of a crash - progress of a kind...
glynos Dec 7, 2013
2012b58
Fixed crash in HTTP client, 'get' now only returns an empty response.
glynos Dec 7, 2013
a8bad17
Continued to work on the HTTP client.
glynos Dec 7, 2013
619255a
Updated the client and response classes in order to make a successful…
glynos Dec 8, 2013
ce3bccf
Validated that the HEAD response returns the headers.
glynos Dec 8, 2013
6eb8e03
Lower case enum for HTTP status code for consistency.
glynos Dec 8, 2013
b6c2bee
Updated client errors and added code to allow the processing of a res…
glynos Dec 8, 2013
8f5ea83
Re-added and updated HTTP examples.
glynos Dec 8, 2013
5d8d744
Added .clang-format for source code formatting.
glynos Dec 8, 2013
6441091
Allow the v0.x HTTP server to be built; Refactored HTTP v2 client req…
glynos Dec 11, 2013
d5b65ee
Refactored client connections into a client_connection namespace.
glynos Dec 11, 2013
23ef8d0
Fixed bug in parsing the HTTP headers in the HTTP client.
glynos Dec 11, 2013
6830eba
Refactored internals of client.
glynos Dec 13, 2013
6ff897f
Continued to refactor the HTTP client; made the method enum class low…
glynos Dec 14, 2013
1238f44
Continued to refactor and document the HTTP client source and tests.
glynos Dec 14, 2013
3e33f13
Continued refactoring, improved read_headers and simple_wget examples.
glynos Dec 14, 2013
a465cd8
Updated uri submodule.
glynos Dec 16, 2013
49310b1
Added a flag to turn on/off the feature tests, which may access the n…
glynos Dec 16, 2013
ebc7762
Added .travis.yml
glynos Dec 16, 2013
e639f47
Fixed some warnings in the HTTP client request and client_exception.
glynos Dec 16, 2013
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updated client errors and added code to allow the processing of a res…
…ponse body.
  • Loading branch information
glynos committed Dec 8, 2013
commit b6c2bee349d91964ac59c39bb9424b7fc376cfb2
77 changes: 62 additions & 15 deletions http/src/http/v2/client/client.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ namespace network {
std::size_t bytes_read,
std::shared_ptr<response> res);

void read_response_body(const boost::system::error_code &ec,
std::size_t bytes_read,
std::shared_ptr<response> res);

std::future<response> do_request(method method_, request request_, request_options options);

client_options options_;
Expand Down Expand Up @@ -83,7 +87,7 @@ namespace network {
if (ec) {
if (endpoint_iterator == tcp::resolver::iterator()) {
response_promise_.set_exception(std::make_exception_ptr(
connection_error(client_error::host_not_found)));
client_exception(client_error::host_not_found)));
return;
}

Expand Down Expand Up @@ -142,19 +146,19 @@ namespace network {
}

std::istream is(&response_);
std::string version;
string_type version;
is >> version;
unsigned int status;
is >> status;
std::string message;
string_type message;
std::getline(is, message);

res->set_version(version);
res->set_status(network::http::v2::status::code(status));
res->set_status_message(boost::trim_copy(message));

connection_->async_read_until(response_,
"\r\n",
"\r\n\r\n",
strand_.wrap(
[=] (const boost::system::error_code &ec,
std::size_t bytes_read) {
Expand All @@ -173,23 +177,65 @@ namespace network {

// fill headers
std::istream is(&response_);
std::string header;
while ((header != "\r") && std::getline(is, header)) {
string_type header;
while (std::getline(is, header) && (header != "\r")) {
std::vector<string_type> kvp;
boost::split(kvp, header, boost::is_any_of(":"));
res->add_header(kvp[0], boost::trim_copy(kvp[1]));
}

connection_->async_read_until(response_,
"\r\n\r\n",
strand_.wrap(
[=] (const boost::system::error_code &ec,
std::size_t bytes_read) {
// um...
response_promise_.set_value(*res);
}));
connection_->async_read(response_,
strand_.wrap(
[=] (const boost::system::error_code &ec,
std::size_t bytes_read) {
read_response_body(ec, bytes_read, res);
}));
}

namespace {
std::istream &getline_with_newline(std::istream &is, std::string &line) {
line.clear();

std::istream::sentry se(is, true);
std::streambuf *sb = is.rdbuf();

while (true) {
int c = sb->sbumpc();
switch (c) {
case EOF:
if (line.empty()) {
is.setstate(std::ios::eofbit);
}
return is;
default:
line += static_cast<char>(c);
}
}
}
} // namespace

void client::impl::read_response_body(const boost::system::error_code &ec,
std::size_t bytes_read,
std::shared_ptr<response> res) {
if (bytes_read == 0) {
response_promise_.set_value(*res);
return;
}

std::istream is(&response_);
string_type line;
while (!getline_with_newline(is, line).eof()) {
res->append_body(line);
}

connection_->async_read(response_,
strand_.wrap(
[=] (const boost::system::error_code &ec,
std::size_t bytes_read) {
read_response_body(ec, bytes_read, res);
}));
}

std::future<response> client::impl::do_request(method met,
request req,
request_options options) {
Expand All @@ -199,7 +245,8 @@ namespace network {
std::ostream request_stream(&request_);
request_stream << req;
if (!request_stream) {
// set error
response_promise_.set_exception(std::make_exception_ptr(
client_exception(client_error::invalid_request)));
}

// HTTP 1.1
Expand Down
21 changes: 8 additions & 13 deletions http/src/http/v2/client/client_errors.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,12 @@ namespace network {
switch (client_error(ev)) {
case client_error::invalid_url:
return "Requires HTTP or HTTPS URL.";
case client_error::resolver_error:
return "Unable to resolve host";
case client_error::invalid_request:
return "Invalid HTTP request.";
case client_error::host_not_found:
return "Unable to resolve host.";
case client_error::invalid_response:
return "Invalid HTTP response.";
default:
break;
}
Expand All @@ -63,21 +67,12 @@ namespace network {

}

resolver_error::resolver_error()
: std::system_error(make_error_code(client_error::resolver_error)) {

}

resolver_error::~resolver_error() {

}

connection_error::connection_error(client_error error)
client_exception::client_exception(client_error error)
: std::system_error(make_error_code(error)) {

}

connection_error::~connection_error() {
client_exception::~client_exception() {

}

Expand Down
60 changes: 9 additions & 51 deletions http/src/network/http/v2/client/client_errors.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ namespace network {
namespace v2 {
/**
* \ingroup http_client
* \enum client_error network/http/v2/client/client_errors.hpp
* \enum client_error network/http/v2/client/client_errors.hpp network/http/v2/client.hpp
* \brief An enumeration of all types of client error.
*/
enum class client_error {
// url
invalid_url,

// resolution
resolver_error,
// request
invalid_request,

// connection
host_not_found,
Expand All @@ -49,7 +49,7 @@ namespace network {

/**
* \ingroup http_client
* \class invalid_url network/http/v2/client/client_errors.hpp
* \class invalid_url network/http/v2/client/client_errors.hpp network/http/v2/client.hpp
* \brief An exception thrown if the URL provides is invalid.
*/
class invalid_url : public std::system_error {
Expand All @@ -70,67 +70,25 @@ namespace network {

/**
* \ingroup http_client
* \class resolver_error network/http/v2/client/client_errors.hpp
* \brief An exception thrown when there is a resolver error.
* \class client_exception network/http/v2/client/client_errors.hpp network/http/v2/client.hpp
* \brief An exception thrown when there is a client error.
*/
class resolver_error : std::system_error {
class client_exception : public std::system_error {

public:

/**
* \brief Constructor.
*/
explicit resolver_error();
explicit client_exception(client_error error);

/**
* \brief Destructor.
*/
virtual ~resolver_error() noexcept;
virtual ~client_exception() noexcept;

};

/**
* \ingroup http_client
* \class connection_error network/http/v2/client/client_errors.hpp
* \brief An exception thrown when there is a connection error.
*/
class connection_error : public std::system_error {

public:

/**
* \brief Constructor.
* \param The client_error code.
*/
explicit connection_error(client_error error);

/**
* \brief Destructor.
*/
virtual ~connection_error() noexcept;

};

/**
* \ingroup http_client
* \class response_error network/http/v2/client/client_errors.hpp
*/
class response_error : public std::system_error {

public:

/**
* \brief Constructor.
* \param The client_error code.
*/
explicit response_error(client_error error);

/**
* \brief Destructor.
*/
virtual ~response_error() noexcept;

};
} // namespace v2
} // namespace http
} // namespace network
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ namespace network {
const std::string &delim,
read_callback callback) = 0;

virtual void async_read(boost::asio::streambuf &command_streambuf,
read_callback callback) = 0;

/**
* \brief Cancels an operation on a connection.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#define NETWORK_HTTP_V2_CLIENT_CONNECTION_NORMAL_CONNECTION_INC

#include <boost/asio/write.hpp>
#include <boost/asio/read.hpp>
#include <boost/asio/read_until.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/io_service.hpp>
Expand Down Expand Up @@ -50,6 +51,12 @@ namespace network {
boost::asio::async_read_until(*socket_, command_streambuf, delim, callback);
}

virtual void async_read(boost::asio::streambuf &command_streambuf,
read_callback callback) {
boost::asio::async_read(*socket_, command_streambuf,
boost::asio::transfer_at_least(1), callback);
}

virtual void cancel() {
socket_->cancel();
}
Expand Down
28 changes: 19 additions & 9 deletions http/src/network/http/v2/client/response.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,8 @@ namespace network {
: version_(other.version_)
, status_(other.status_)
, status_message_(other.status_message_)
, headers_(other.headers_) {
, headers_(other.headers_)
, body_(other.body_) {

}

Expand All @@ -79,7 +80,8 @@ namespace network {
: version_(std::move(other.version_))
, status_(std::move(other.status_))
, status_message_(std::move(other.status_message_))
, headers_(std::move(other.headers_)) {
, headers_(std::move(other.headers_))
, body_(std::move(other.body_)) {

}

Expand All @@ -102,13 +104,14 @@ namespace network {
swap(status_, other.status_);
swap(status_message_, other.status_message_);
swap(headers_, other.headers_);
swap(body_, other.body_);
}

/**
* \brief Sets the HTTP version.
* \param version The HTTP version (1.0 or 1.1).
*/
void set_version(const string_type &version) {
void set_version(string_type version) {
version_ = version;
}

Expand Down Expand Up @@ -140,7 +143,7 @@ namespace network {
* \brief Sets the HTTP response status message.
* \param status The HTTP response status message.
*/
void set_status_message(const string_type &status_message) {
void set_status_message(string_type status_message) {
status_message_ = status_message;
}

Expand All @@ -163,19 +166,26 @@ namespace network {
return boost::make_iterator_range(std::begin(headers_), std::end(headers_));
}

std::future<string_type> read_body(std::size_t length) const;
void append_body(const char *body, std::size_t length) {
body_.reserve(body_.size() + length);
body_.append(body);
}

void append_body(string_type body) {
body_.append(body);
}

// add_header
// set_body
// append_body
// get_body
string_type body() const {
return body_;
}

private:

string_type version_;
network::http::v2::status::code status_;
string_type status_message_;
headers_type headers_;
string_type body_;

};

Expand Down