How to Import multiple CSV Files From S3 at Once

Install aws.s3 package

From Project Header menu, select Manage R Packages.

Type in aws.s3 and click install.

Once it's done, it shows "successfully installed" message.

Import CSV Files with R Script Data Source

Click the plus button next to Data Frames and select R Script.

Below example explains a way to import multiple CSV files whose name looks like "<some_string>_sample.csv" as a single data frame from a AWS S3 Bucket.

library(aws.s3)
library(purrr)
Sys.setenv("AWS_ACCESS_KEY_ID" = "YOUR_ACCESS_KEY_ID",
           "AWS_SECRET_ACCESS_KEY" = "YOUR_ACCESS_KEY",
           "AWS_DEFAULT_REGION" = "YOUR_REGION_LIKE_us-west-2")

csvs <- get_bucket_df("your_bucket", max= Inf) %>% filter(str_detect(Key, "sample.*\\.csv$")) 

map_df(csvs$Key, function(key){
  read_csv(get_object(key, bucket = "your_bucket", as = "text"))
})

Make sure to set AWS Access Key ID, AWS Secret Access Key, Region, and Bucket with appropriate values. If your bucket contains more than 1,000 files, you want to pass Max = Inf as the argument of get_bucket_df.

Click Save to save it as a data frame.