Recent commits on myprojects-hg-reborn (hg) - larry77 - OSDN https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/ e7bfcf: I improved the fruit repository. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/e7bfcf3fb5934fc4384aa0e6218c2eda17f278d0 Lorenzo Isella Commit: e7bfcf3fb5934fc4384aa0e6218c2eda17f278d0
I improved the fruit repository.
]]>
c30b26: I added new entries of my sources.list.d directory. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/c30b264bd21cb3b164fcd9b9c19b57f25432376d Lorenzo Isella Commit: c30b264bd21cb3b164fcd9b9c19b57f25432376d
I added new entries of my sources.list.d directory.
]]>
32fb8f: I added a file to configure mpv. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/32fb8fd497e6e63018b63e2c8a0d251e43a5e635 Lorenzo Isella Commit: 32fb8fd497e6e63018b63e2c8a0d251e43a5e635
I added a file to configure mpv.
]]>
e33ee7: I now save the parquet files with a more expressive name. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/e33ee7191ce5974e1609b81bea2eef0e3f858ec0 Lorenzo Isella Commit: e33ee7191ce5974e1609b81bea2eef0e3f858ec0
I now save the parquet files with a more expressive name.
]]>
1b78ef: The solution with write_dataset is better when working wi... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/1b78ef2e4c2db45ef5acb4d956961ef64356785d Lorenzo Isella Commit: 1b78ef2e4c2db45ef5acb4d956961ef64356785d
The solution with write_dataset is better when working with very large files. I write multiple parquet files and I have no ram issues at all.
]]>
e8eca9: I added a new code showing how to handle a big csv file, ... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/e8eca9502916827e3f4da6caa9ca8d862f49bbf0 Lorenzo Isella Commit: e8eca9502916827e3f4da6caa9ca8d862f49bbf0
I added a new code showing how to handle a big csv file, giving it an automatic schema and filter the results.
]]>
305074: I now ensure that I remove final_spain.csv before I gener... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/305074685d9864dab8a122477df3747ec1f32740 Lorenzo Isella Commit: 305074685d9864dab8a122477df3747ec1f32740
I now ensure that I remove final_spain.csv before I generate it.
]]>
c8d552: I added another file to carry out some diagnostics on TAM. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/c8d552ffbe18af1d7cabba1f693821695f9f75f0 Lorenzo Isella Commit: c8d552ffbe18af1d7cabba1f693821695f9f75f0
I added another file to carry out some diagnostics on TAM.
]]>
f57b05: I made a mistake with the saving of a compressed csv file. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/f57b0540403c2d9e24fad98e45fd63c2e847f7a8 Lorenzo Isella Commit: f57b0540403c2d9e24fad98e45fd63c2e847f7a8
I made a mistake with the saving of a compressed csv file.
]]>
0eeeca: I modified the obfuscated data set. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/0eeeca691bae73b56cb039d6881c5d01ea5b26ff Lorenzo Isella Commit: 0eeeca691bae73b56cb039d6881c5d01ea5b26ff
I modified the obfuscated data set.
]]>
4e6f14: I save the data directly as a compressed csv file. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/4e6f14cc0e4e90af75c07a26f3dc54d0047bd68a Lorenzo Isella Commit: 4e6f14cc0e4e90af75c07a26f3dc54d0047bd68a
I save the data directly as a compressed csv file.
]]>
81e6b8: I changed the way in which I handle the date. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/81e6b8181ec62c4d133979c33c56bd940891fbe0 Lorenzo Isella Commit: 81e6b8181ec62c4d133979c33c56bd940891fbe0
I changed the way in which I handle the date.
]]>
036b6d: I changed the file names and I now use write_csv_arrow to... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/036b6d674044e9c0782a61541969345e7fa549ea Lorenzo Isella Commit: 036b6d674044e9c0782a61541969345e7fa549ea
I changed the file names and I now use write_csv_arrow to save the data as a csv.
]]>
0a117d: I now can choose to skip the first part of the data proce... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/0a117d28cd277fba0dbd193dfbcf4e372336ad59 Lorenzo Isella Commit: 0a117d28cd277fba0dbd193dfbcf4e372336ad59
I now can choose to skip the first part of the data processing once it has been done for all.
]]>
8fadfc: I now use write_parquet instead of write_dataset to save ... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/8fadfc840e062f275f21c132d082a225f30e93c2 Lorenzo Isella Commit: 8fadfc840e062f275f21c132d082a225f30e93c2
I now use write_parquet instead of write_dataset to save the results. I am not interested in a multifile solution.
]]>
faae68: I changed the name of the input file. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/faae683130e0949f235b0b0607391485529d12f0 Lorenzo Isella Commit: faae683130e0949f235b0b0607391485529d12f0
I changed the name of the input file.
]]>
2cb03c: I remove two columns I do not need any longer. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/2cb03c06206d4a1e8ad90512f28d80d48ce1d8bf Lorenzo Isella Commit: 2cb03c06206d4a1e8ad90512f28d80d48ce1d8bf
I remove two columns I do not need any longer.
]]>
1e5dfb: I modified the code because of modifications in the Slove... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/1e5dfb3a027a9c9bef92ee47e0f15bdb99b93b48 Lorenzo Isella Commit: 1e5dfb3a027a9c9bef92ee47e0f15bdb99b93b48
I modified the code because of modifications in the Slovenian input files.
]]>
5f02e8: I cleaned up the code and I also generate a parquet file. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/5f02e8dbea0d074338c3042c14ad48d3e5a4a71a Lorenzo Isella Commit: 5f02e8dbea0d074338c3042c14ad48d3e5a4a71a
I cleaned up the code and I also generate a parquet file.
]]>
934169: A better way to handle the data: I process individually e... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/93416964f8e9abc8f1fd5eb1c6f344bb948df93b Lorenzo Isella Commit: 93416964f8e9abc8f1fd5eb1c6f344bb948df93b
A better way to handle the data: I process individually each page file retrieved from the Spanish state aid site.
]]>
4a2814: A new code to process the Spanish data obtained via the API. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/4a28148024e7860c2a2d3d0e6218c66781d3933c Lorenzo Isella Commit: 4a28148024e7860c2a2d3d0e6218c66781d3933c
A new code to process the Spanish data obtained via the API.
]]>
67eb8c: Minor modifications. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/67eb8cf5c5498ab23c35ce92181fa2be7d8a37f5 Lorenzo Isella Commit: 67eb8cf5c5498ab23c35ce92181fa2be7d8a37f5
Minor modifications.
]]>
9f0da8: A simple script to query the tam. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/9f0da811c8045dc8ddf01b9d09530fa6b3d874e3 Lorenzo Isella Commit: 9f0da811c8045dc8ddf01b9d09530fa6b3d874e3
A simple script to query the tam.
]]>
a79c6e: I fixed some typos. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/a79c6ee253aca6e17b9817a7f86cb2da99c487cc Lorenzo Isella Commit: a79c6ee253aca6e17b9817a7f86cb2da99c487cc
I fixed some typos.
]]>
307677: I also create the railway dataset. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/30767703d899a1539cde3fdcac90600d0f447a9b Lorenzo Isella Commit: 30767703d899a1539cde3fdcac90600d0f447a9b
I also create the railway dataset.
]]>
1a9c5a: Code to retrieve the Spanish State aid data via their API. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/1a9c5aa872b5c5ad59f5d78a58c8563346376981 Lorenzo Isella Commit: 1a9c5aa872b5c5ad59f5d78a58c8563346376981
Code to retrieve the Spanish State aid data via their API.
]]>
5f4b07: I simply replaced magritte pipe with the native pipe. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/5f4b0742115d6138605cd73c433a3164d6536f34 Lorenzo Isella Commit: 5f4b0742115d6138605cd73c433a3164d6536f34
I simply replaced magritte pipe with the native pipe.
]]>
26cf26: I fixed a function to calculate the lags. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/26cf26bd8ced9579691cb639e95c74b520791870 Lorenzo Isella Commit: 26cf26bd8ced9579691cb639e95c74b520791870
I fixed a function to calculate the lags.
]]>
9d96c6: I added new indicators. https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/9d96c62b1253024f91db412d6aa8daf07ac6ae61 Lorenzo Isella Commit: 9d96c62b1253024f91db412d6aa8daf07ac6ae61
I added new indicators.
]]>
f250f2: I now save also time-stamped versions of the portfolio co... https://zh.osdn.net/users/larry77/pf/myprojects-hg-reborn/scm/commits/f250f238e4adec02ada03b9624a70d8e00021fd6 Lorenzo Isella Commit: f250f238e4adec02ada03b9624a70d8e00021fd6
I now save also time-stamped versions of the portfolio compositions as pdf files.
]]>