提交: c1a8b701554dd8371c2d9274a2ea4d79ae3bf908 - myprojects-hg-reborn (hg)

Commit MetaInfo

• 修订版: c1a8b701554dd8371c2d9274a2ea4d79ae3bf908 (tree)
• 时间: 2025-01-01 02:33:41
• 作者: Lorenzo Isella <lorenzo.isella@gmai...>
• Commiter: Lorenzo Isella

Log Message

I added a script showcasing how to work on a csv file both with arrow and duckplyr.

更改概述

• add: R-codes/duckplyr_test.R

差异

diff -r 646bddfa8bc1 -r c1a8b701554d R-codes/duckplyr_test.R
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/R-codes/duckplyr_test.R	Tue Dec 31 18:33:41 2024 +0100
@@ -0,0 +1,40 @@
+rm(list=ls())
+library(tidyverse)
+library(duckplyr)
+library(tictoc)
+library(arrow)
+
+
+# Increased the size of data
+## dd <- tibble(x=1:100000000, y=rep(LETTERS[1:20], 5000000))
+
+
+## write_csv(dd, "test.csv")
+
+
+df <- duck_csv("test.csv")
+
+system.time({
+df_stat <- df |>
+    summarise(total=sum(x), .by = y) |> 
+    collect() |>
+    as_tibble()
+
+})
+
+
+df2  <- open_dataset("test.csv",
+  format = "csv",
+  skip_rows = 0)
+
+system.time({
+    df_stat2 <- df2 |>
+    group_by(y) |> 
+    summarise(total=sum(x)) |>
+    ungroup() |> 
+    collect()
+
+})
+
+
+print("So far so good")