Easy to use webpki-roots #575

joshtriplett · 2022-07-06T05:55:19Z

Describe the bug

aws-sdk-rust includes an example of how to configure hyper-rustls to use webpki-roots and TLS 1.3, in examples/tls/src/lib.rs.

However, this example has two separate versions of hyper-rustls involved (0.23.0 and 0.22.1), and while hyper-rustls 0.23.0 is configured to use webpki-roots, something in aws-sdk-rust is still attempting to initialize hyper-rustls 0.22.1, which is failing with this error:

thread 'main' panicked at 'no CA certificates found', /home/josh/.cargo/registry/src/suiyiyu.us.kg-1ecc6299db9ec823/hyper-rustls-0.22.1/src/connector.rs:45:13

(The easiest way to test this is to build a static binary using the x86_64-unknown-linux-musl target and run it in an otherwise empty chroot.)

Expected Behavior

I'd like to use aws-sdk-rust with only webpki-roots, in a standalone static binary that has no system certificate store.

Current Behavior

thread 'main' panicked at 'no CA certificates found', /home/josh/.cargo/registry/src/suiyiyu.us.kg-1ecc6299db9ec823/hyper-rustls-0.22.1/src/connector.rs:45:13

Reproduction Steps

Here's a cut down version of the example (skipping the TLS 1.3 bits):

`Cargo.toml`

[package]
name = "awstest"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = "1.0.58"
async-std = { version = "1.12.0", features = ["tokio1", "attributes"] }
aws-config = { version = "0.15.0", default-features = false, features = ["rustls"] }
aws-sdk-sts = { version = "0.15.0", default-features = false }
aws-smithy-client = { version = "0.45.0", default-features = false }
hyper-rustls = { version = "0.23.0", default-features = false, features = ["http1", "http2", "tls12", "webpki-roots", "webpki-tokio"] }

(aws-config needs rustls enabled due to an unrelated bug. And the hyper-rustls dependency is coming in via aws-smithy-client: despite default-features = false, there's an indirect dependency on aws-smithy-client's default features via aws-types, among other things.)

`src/main.rs`

#[async_std::main]
async fn main() -> anyhow::Result<()> {
    let https_connector = hyper_rustls::HttpsConnectorBuilder::new()
        .with_webpki_roots()
        .https_only()
        .enable_http1()
        .enable_http2()
        .build();
    let provider_config = aws_config::provider_config::ProviderConfig::default()
        .with_tcp_connector(https_connector.clone())
        .load_default_region()
        .await;
    let aws_sdk_config = aws_config::from_env()
        .configure(provider_config)
        .load()
        .await;

    let sts = aws_sdk_sts::Client::from_conf_conn(
        aws_sdk_sts::Config::from(&aws_sdk_config),
        aws_smithy_client::hyper_ext::Adapter::builder().build(https_connector),
    );

    let identity = sts.get_caller_identity().send().await?;
    dbg!(identity);
    Ok(())
}

Some debug prints suggest that the issue happens when trying to create aws_sdk_config. The example in "possible solution" gives a workaround.

Possible Solution

Alternate example that does work:

`src/main.rs`

#[async_std::main]
async fn main() -> anyhow::Result<()> {
    let https_connector = hyper_rustls::HttpsConnectorBuilder::new()
        .with_webpki_roots()
        .https_only()
        .enable_http1()
        .enable_http2()
        .build();
    let provider_config = aws_config::provider_config::ProviderConfig::default()
        .with_tcp_connector(https_connector.clone())
        .load_default_region()
        .await;
    let adapter = aws_smithy_client::hyper_ext::Adapter::builder().build(https_connector);
    let dyn_connector = aws_smithy_client::erase::DynConnector::new(adapter.clone());
    let aws_sdk_config = aws_config::from_env()
        .http_connector(aws_smithy_client::http_connector::HttpConnector::Prebuilt(Some(dyn_connector)))
        .configure(provider_config)
        .load()
        .await;

    let sts = aws_sdk_sts::Client::from_conf_conn(
        aws_sdk_sts::Config::from(&aws_sdk_config),
        adapter,
    );

    let identity = sts.get_caller_identity().send().await?;
    dbg!(identity);
    Ok(())
}

Additional Information/Context

No response

Version

├── aws-config v0.15.0
│   ├── aws-http v0.15.0
│   │   ├── aws-smithy-http v0.45.0
│   │   │   ├── aws-smithy-types v0.45.0
│   │   ├── aws-smithy-types v0.45.0 (*)
│   │   ├── aws-types v0.15.0
│   │   │   ├── aws-smithy-async v0.45.0
│   │   │   ├── aws-smithy-client v0.45.0
│   │   │   │   ├── aws-smithy-async v0.45.0 (*)
│   │   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   │   ├── aws-smithy-http-tower v0.45.0
│   │   │   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   │   ├── aws-smithy-types v0.45.0 (*)
│   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   ├── aws-smithy-types v0.45.0 (*)
│   ├── aws-sdk-sso v0.15.0
│   │   ├── aws-endpoint v0.15.0
│   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   ├── aws-types v0.15.0 (*)
│   │   ├── aws-http v0.15.0 (*)
│   │   ├── aws-sig-auth v0.15.0
│   │   │   ├── aws-sigv4 v0.15.0
│   │   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   ├── aws-smithy-http v0.45.0 (*)
│   │   │   ├── aws-types v0.15.0 (*)
│   │   ├── aws-smithy-async v0.45.0 (*)
│   │   ├── aws-smithy-client v0.45.0 (*)
│   │   ├── aws-smithy-http v0.45.0 (*)
│   │   ├── aws-smithy-http-tower v0.45.0 (*)
│   │   ├── aws-smithy-json v0.45.0
│   │   │   └── aws-smithy-types v0.45.0 (*)
│   │   ├── aws-smithy-types v0.45.0 (*)
│   │   ├── aws-types v0.15.0 (*)
│   ├── aws-sdk-sts v0.15.0
│   │   ├── aws-endpoint v0.15.0 (*)
│   │   ├── aws-http v0.15.0 (*)
│   │   ├── aws-sig-auth v0.15.0 (*)
│   │   ├── aws-smithy-async v0.45.0 (*)
│   │   ├── aws-smithy-client v0.45.0 (*)
│   │   ├── aws-smithy-http v0.45.0 (*)
│   │   ├── aws-smithy-http-tower v0.45.0 (*)
│   │   ├── aws-smithy-query v0.45.0
│   │   │   ├── aws-smithy-types v0.45.0 (*)
│   │   ├── aws-smithy-types v0.45.0 (*)
│   │   ├── aws-smithy-xml v0.45.0
│   │   ├── aws-types v0.15.0 (*)
│   ├── aws-smithy-async v0.45.0 (*)
│   ├── aws-smithy-client v0.45.0 (*)
│   ├── aws-smithy-http v0.45.0 (*)
│   ├── aws-smithy-http-tower v0.45.0 (*)
│   ├── aws-smithy-json v0.45.0 (*)
│   ├── aws-smithy-types v0.45.0 (*)
│   ├── aws-types v0.15.0 (*)
├── aws-sdk-sts v0.15.0 (*)
├── aws-smithy-client v0.45.0 (*)

Environment details (OS name and version, etc.)

Debian, latest sid

Logs

No response

The text was updated successfully, but these errors were encountered:

Velfi · 2022-07-06T16:06:40Z

We are working on reducing the number of branching feature paths. We plan to accomplish this by unifying the HTTP connection providers and possibly providing them through a separate crate.

Ultimately, we want to avoid the situation where people (including us) have to understand complex branching rules in order to understand the ways that HTTP/HTTPS client creation and configuration happens.

joshtriplett · 2022-07-08T05:34:21Z

Ultimately, we want to avoid the situation where people (including us) have to understand complex branching rules in order to understand the ways that HTTP/HTTPS client creation and configuration happens.

I can definitely appreciate that. My primary goal with this was simplicity as well; I don't just want to reduce dependencies (which I care about as well), but also, omitting native certificate support entirely seems like the easiest way to be confident that no possible code path will attempt to read them.

Nonetheless, I'd be happy with any solution path that makes that possible, including a refactor to do HTTP initialization in a different way. I'd just like to make sure that it's not substantially harder to run with webpki-roots than to run with native roots.

Thank you for aws-sdk-rust!

rcoh · 2022-07-08T18:30:11Z

In either case, I think there's a bug here in aws-config where something isn't building a connector properly. Thanks for the reproducer, we'll track this down

joshtriplett · 2023-05-01T02:45:44Z

Checking back in on this: it still seems like there's no way to configure the built-in usage of hyper-rustls to use webpki-roots, and the only solution is overriding it completely and making sure the built-in version never gets invoked. I'd love to have a simpler and more robust solution than that.

jmklix · 2024-03-15T17:21:58Z

Changing this to feature request to make webpki-roots easier to use.

joshtriplett added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 6, 2022

Velfi removed the needs-triage This issue or PR still needs to be triaged. label Jul 6, 2022

Velfi mentioned this issue Jul 7, 2022

aws-smithy-client: Support using rustls with either native roots or webpki roots smithy-lang/smithy-rs#1535

Closed

1 task

jdisanti added the low-priority This issue is low-priority and will be addressed once we have fewer high-priority tasks label Jul 20, 2022

jmklix added p3 This is a minor priority issue and removed low-priority This issue is low-priority and will be addressed once we have fewer high-priority tasks labels Nov 14, 2022

jmklix changed the title ~~Configuration using webpki-roots still breaks if native system certificates unavailable~~ Easy to use webpki-roots Mar 15, 2024

jmklix added feature-request A feature should be added or improved. and removed bug This issue is a bug. labels Mar 15, 2024

flisky mentioned this issue Jul 11, 2024

customize roots #1171

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Easy to use webpki-roots #575

Easy to use webpki-roots #575

joshtriplett commented Jul 6, 2022 •

edited

Loading

Velfi commented Jul 6, 2022

joshtriplett commented Jul 8, 2022

rcoh commented Jul 8, 2022

joshtriplett commented May 1, 2023

jmklix commented Mar 15, 2024

Easy to use webpki-roots #575

Easy to use webpki-roots #575

Comments

joshtriplett commented Jul 6, 2022 • edited Loading

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Cargo.toml

src/main.rs

Possible Solution

src/main.rs

Additional Information/Context

Version

Environment details (OS name and version, etc.)

Logs

Velfi commented Jul 6, 2022

joshtriplett commented Jul 8, 2022

rcoh commented Jul 8, 2022

joshtriplett commented May 1, 2023

jmklix commented Mar 15, 2024

joshtriplett commented Jul 6, 2022 •

edited

Loading

`Cargo.toml`

`src/main.rs`

`src/main.rs`