Science_blog: February 2023

Tuesday, 28 February 2023

How to insert date in row in R

#########Import your file#########

> culti <- read.csv("cultivation2.csv", header = TRUE, sep = ",")

> culti$date <- as.Date(culti$hdate)

> cult2 <- data.frame(culti$date, culti$site, culti$code)

> colnames(cult2) <- c("date", "site", "crop")

> cutiec1 <- cult2 %>%

filter(site == 1)

###########Do not forget to check your structure of data#######

> str(cutiec1) ("It should be character otherwise it would not work")

'data.frame': 11 obs. of 3 variables:

$ date: Date, format: "2009-12-12" ...

$ site: int 1 1 1 1 1 1 1 1 1 1 ...

$ crop: chr "CC" "SM" "WW" "WR" ...

####Change date to character format all rows

> cutiec1$date <- as.character(cutiec1$date)

> cutiec1$site <- as.character(cutiec1$site)

###option 1#####

> cutiec1[nrow(cutiec1)+ 1, ] <- c("2018-09-29")

but this would create problems like this

date site crop

1 2009-08-02 1 <NA>

2 2009-12-12 1 CC

3 2010-10-14 1 SM

4 2011-07-28 1 WW

5 2012-07-20 1 WR

6 2013-08-04 1 WW

7 2013-12-11 1 CC

8 2014-10-09 1 SM

9 2015-07-22 1 WW

10 2016-11-01 1 GM

11 2017-07-30 1 WW

12 2018-07-09 1 WR

13 2018-09-29 2018-09-29 2018-09-29

Hence follow this ---

############Option 2##########

> cutiec1 <- cutiec1 %>% add_row(date="2009-08-02", site="1", .before = 1)

> cutiec1 <- cutiec1 %>% add_row(date="2018-09-29", site="1", .after = 12)

######then character date to POSIXct date

> cutiec1$date <- as.POSIXct(cutiec1$date, tz = "UTC")

#########Then convert POSIXct date to date format

> cutiec1$date <- as.Date(cutiec1$date)

Then do "pad". It will creat date from 2009-08-02 to 2018-09-29 at interval of day

> cutiec12 <- pad(cutiec1, interval = "day",)

It shows 3346 variables for all 9 years on daily basis.

#############################Done##############################################

Monday, 27 February 2023

Mitigation approaches in agriculture

In my opinion, followings could be the mitigation approaches in agriculture engineering

1. Reducing N2O emissions

2. Reducing leaching

3. Carbon captured and storage

4. No tillage

5. Plant breeding

6. Agroforestry

7. Restoration of forest

8. wetland

9. Reducing dryland area

Monday, 20 February 2023

ggarrange: Plotting with common axis in R

##########Import your file####################################

> setwd("/home/rajiv/for_1695_datast_confidential/04_weather")

> ec1w <- read.csv("ec1_weather_data.csv", header = TRUE, sep = ",")

> head(ec1w)

###Convert date to as date with column "Date"

> ec1w$Date <- as.Date(ec1w$date)

############How to plot graph in ggplot2######################

1. > install.packages("ggplot2")

> library('ggplot2')

###plotting graph with ggplot2

> ggplot2(ec4w)+

geom_line(aes(x=Date, y=pr))

2. > install.packages("gtidyverse")

> library('tidyverse')

3. > install.packages("ggpubr",

repos = c("https://cran.rediris.org/", "https://cloud.r-project.org/"),

dependencies = TRUE)

> library('ggpubr)

##############ploting with ggarrange

> ec1 <- ggplot2(dataframe1)+

geom_line(aes(x=Date, y=at))+

geom_col(aes(x=Date, y=pr), color = "red")

> ec2 <- ggplot2(dataframe2)+

geom_line(aes(x=Date, y=at))+

geom_col(aes(x=Date, y=pr), color = "red")

> ec3 <- ggplot2(dataframe3)+

geom_line(aes(x=Date, y=at))+

geom_col(aes(x=Date, y=pr), color = "red")

> ggpubr::ggarrange(ec1, ec2, ec3,

ncol = 1, nrow = 3)

#################Plotting with common axis with ggarrange

> library(grid)

> library(gridExtra)

> ec111 <- ggpubr::ggarrange(ec1 + rremove("ylab") + rremove("xlab"),

ec2 + rremove("ylab") + rremove("xlab"),

ec3 + rremove("ylab") + rremove("xlab"),

labels = c("EC1",

"EC2",

"EC3"),

ncol = 1,

nrow = 3,

hjust = -15,

common.legend = TRUE,

legend = "bottom",

align = "hv",

font.label = list(size = 10, color = "black", face = "bold", family = NULL, position = "top"))

then,

> annotate_figure(ec111, left = textGrob("Temperature (black) and Precipitation (red)", rot = 90, vjust = 1, gp = gpar(cex = 1.3)),

bottom = textGrob("Date", gp = gpar(cex = 1.3)))

I made this with my data.

Sunday, 19 February 2023

Linux Vs Macos: Which is better

Let's talk about Linux Vs Macos. Something is more interesting and amazing. As, you may have noticed that the data science/engineering people works on either Linux or Macos and most high performance computing systems use Linux version because it allow all programming software while in widows, you need platform and they are slower as much and the reason is that all core are busy in other work and it allow one core only for that software and rest for windows. But, Macos allow you for other software too as linux and it comes with many. Hence, now days it demand has been increased to for data science. However, both (Linux and Macos) share Unix management system. Now the question is which one is strong ?

1. Open source vs Proprietary software

Both Linux and Macos use open source software system Unix. But, Linux distribution is entirely based on free software while part of Macos use of open source and rest are proprietary based for that you have to pay. Since, Linux is based on open source then you edit and modify if you are specialist in software work and still a freely robust distribution.

2. Desktop environment

At the early stage, Linux did not come with desktop environment instead you have to use access with terminal. But, now a days, it allows many desktop environment i.e. Ubuntu, fedora and then with GNOME, XFCE, KDE, Deepin and many more.

GUI in Macos is standard and same for all customers or users and nothing new.

3. Hardware

You can install Linux on computers with any configurations. Even, you can install on Windows and Macos too by sing virtual machine and it require less space than others.

But with Macos, fortunately not after all it paid service.

4. Security

I think there is no any software which is 100% secure even Linux too and reason is very simple. It is free for everyone even for hackers too so they can develop tool to spoil it and may be they are trying but the chances is less in compression too windows and Macos and reason is these two are not free and currently, they are on top to earn money.

However, with Linux, it is less chance and reason is free so if someone will hack my computer i can format computer and reinstall and another strong which make Linux system strong than others is commend line system (CLI). Hence, to access any files in systems to need permission every time this is reason why auto run type virsus could not affect any more.

And, another advantage, Linux is open source then there is no way to hide malicious or privacy violating code. Any one can remove it out.

5. Professional Carrier

Yes, it can help it to make your carrier in computer science and data science even more. As i said most data company and tech company prefer Linux system than the windows and Macos and if you are familiar with this there is high chance to select you. I have seen most high performance computing system prefer only Linux.

The only good point with macos that it also come with CLI or terminal based hence you can use it for that purpose but if other system don't have.

Conclusions

1. For carrier, Linux is better than Macos

2. Linux is high secure than Macos

3. Macos allow video editing service than the Linux but you have to pay a lot.

4. In terms of price, Linux dominant as it is free.

Suggestion

Whole thing is depend upon you. Do you like to invest money or learning computer system. If you have money don't think too much buy Macos it gives you privacy but if you are at early stage of professional carrier then go for Linux to learn how to use terminal otherwise you could lost in Macos system and except watching video and internate suffering, you can not do any thing and you will say like my friends "Why Macos is so popular and what Macos does". Remember, Macos do not allow neither MS office nor like MS office tools so be careful. It don't allow to access and modify word and excel files and if you want then you have to pay a lot.

Macos allow you to do video, pic, gif editing and creating apps for Macos beyond of this there is Windows. But Linux allows everything what i wrote above. One another point, In Europe, most people prefer Macos for personal use and for Carrier they choose Linux.

Thursday, 16 February 2023

How to change date format in notepad++

You can do this with notepad++:

< >

Find: ([0-9]+)-+([0-9]+)-+([0-9]+)

Replace: \3-\2-\1

Note: Careful with \3-\2-\1. It depends upon your date format what you have wrote in notepad++. It represents the potion of year, date and month

#######

To make sure you only reorder wrong formats (in case you have mixed formats from merging databases), use this:

([0-9]{2})-+([0-9]{2})-+([0-9]{4})

This searches for (four digits, dash, two digits, dash, two digits).

In an regex capable editor like notepad++, replace it with this:

\3-\2-\1

In a tool like libre office, you need to replace it with this:

$3-$2-$1

#############

Used Notepad++ to change mm/dd/yyyy to yyyy/mm/dd in several lines of a text file. Script was saved as a macro for next file.

Find: ([0-9]{2})/+([0-9]{2})/+([0-9]{4}) Replace: \3/\1/\2

#########

Capturing groups of various elements of an ISO date representation

"(\d{4})-(\d{1,2})-(\d{1,2})T([0-2]\d):([0-5]\d):([0-5]\d)(?:.\d+)?Z?\s?"

In total six groups are being captured in this pattern, seventh group in the end containing '?:' is non-capturing, meaning it does not keep any value recorded by the group, simply ignores them.

The six groups we have captured have numbers based on positions such as \1 \2 \3... upto \6.

Replacement pattern to change that date into Database supported date format mostly used in Oracle values like - "11/30/2010 00:00:00 AM"

"\2/\3/\1 \4:\5:\6 AM"

It works well with Notepad++ Good Luck!

Filter row wise data from data frame with dates in R

What Is the Best Way to Filter by Date in R?

Method 1: After Date Filter Rows

df %>% filter(date_column > '2022-01-01')

Method 2: Filter Rows Before Date

df %>% filter(date_column < '2022-01-01')

Method 3: Filter Rows Between Two Dates

df %>% filter(between(date_column, as.Date('2022-01-20'), as.Date('2022-02-20')))

Let’s create a data frame

df <- data.frame(day=seq(as.Date('2022-01-01'), by = 'week', length.out=10),

sales=c(40, 35, 39, 44, 48, 51, 23, 29, 60, 65))

Now we can view the data frame

day sales

1 2022-01-01 240

2 2022-01-08 335

3 2022-01-15 359

4 2022-01-22 544

5 2022-01-29 548

6 2022-02-05 251

7 2022-02-12 223

8 2022-02-19 529

9 2022-02-26 660

10 2022-03-05 165

Example 1: Filter Rows After Date

To filter for rows in the data frame with a date after 1/25/2022, use the following code.

library(tidyverse)

filter for rows with dates after 1/25/2022

df %>% filter(day > '2022-01-25')

day sales

1 2022-01-29 548

2 2022-02-05 251

3 2022-02-12 223

4 2022-02-19 529

5 2022-02-26 660

6 2022-03-05 165

Each row in the generated data frame has a date that is later than 1/25/2022.

Best Books on Data Science with Python – Data Science Tutorials

Example 2: Filter Rows Before Date

To filter for rows in the data frame with a date before 1/25/2022, we can use the following code.

library(dplyr)

Let’s filter for rows with dates before 1/25/2022

df %>% filter(day < '2022-01-25')

day sales

1 2022-01-01 240

2 2022-01-08 335

3 2022-01-15 359

4 2022-01-22 544

Each entry in the generated data frame has a date that is prior to 1/25/2022.

Example 3: Filter Rows Between Two Dates

To filter for rows in the data frame with a date between 1/20/2022 and 2/20/2022, use the following code.

library(tidyverse)

filter for rows with dates between 1/20/2022 and 2/20/2022

Best Data Science YouTube Tutorials Free to Learn – Data Science Tutorials

df %>% filter(between(daty, as.Date('2022-01-20'), as.Date('2022-02-20')))

day sales

1 2022-01-22 544

2 2022-01-29 548

3 2022-02-05 251

4 2022-02-12 223

5 2022-02-19 529

The dates in the rows of the generated data frame range from 1/20/2022 to 2/20/2022.

If none of the ways above work, you may need to use them as.Date() function to convert the dates you’re working with to a recognized date format.

How to install R library 'tidyverse' for updated R version 4.2.1 on latest Ubuntu 22.2

Dear all,

I was trying to install tidyverse in Rv4.2.1 in Ubuntu 22.2 ans whenever i as trying to install tidyverse I got an error "ERROR: configuration failed for package ‘curl’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/curl’

Warning in install.packages :

installation of package ‘curl’ had non-zero exit status

* installing *source* package ‘xml2’ ...

** package ‘xml2’ successfully unpacked and MD5 sums checked

** using staged installation

Package libxml-2.0 was not found in the pkg-config search path.

Perhaps you should add the directory containing `libxml-2.0.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libxml-2.0' found

Package libxml-2.0 was not found in the pkg-config search path.

Perhaps you should add the directory containing `libxml-2.0.pc'

to the PKG_CONFIG_PATH environment variable

No package 'libxml-2.0' found

Using PKG_CFLAGS=

Using PKG_LIBS=-lxml2

------------------------- ANTICONF ERROR ---------------------------

Configuration failed because libxml-2.0 was not found. Try installing:

* deb: libxml2-dev (Debian, Ubuntu, etc)

* rpm: libxml2-devel (Fedora, CentOS, RHEL)

* csw: libxml2_dev (Solaris)

If libxml-2.0 is already installed, check that 'pkg-config' is in your

PATH and PKG_CONFIG_PATH contains a libxml-2.0.pc file. If pkg-config

is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:

R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'

--------------------------------------------------------------------

ERROR: configuration failed for package ‘xml2’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/xml2’

Warning in install.packages :

installation of package ‘xml2’ had non-zero exit status

ERROR: dependency ‘curl’ is not available for package ‘httr’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/httr’

Warning in install.packages :

installation of package ‘httr’ had non-zero exit status

ERROR: dependency ‘httr’ is not available for package ‘gargle’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/gargle’

Warning in install.packages :

installation of package ‘gargle’ had non-zero exit status

ERROR: dependencies ‘httr’, ‘xml2’ are not available for package ‘rvest’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/rvest’

Warning in install.packages :

installation of package ‘rvest’ had non-zero exit status

ERROR: dependencies ‘gargle’, ‘httr’ are not available for package ‘googledrive’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/googledrive’

Warning in install.packages :

installation of package ‘googledrive’ had non-zero exit status

ERROR: dependencies ‘curl’, ‘gargle’, ‘googledrive’, ‘httr’ are not available for package ‘googlesheets4’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/googlesheets4’

Warning in install.packages :

installation of package ‘googlesheets4’ had non-zero exit status

ERROR: dependencies ‘googledrive’, ‘googlesheets4’, ‘httr’, ‘rvest’, ‘xml2’ are not available for package ‘tidyverse’

* removing ‘/home/rajiv/R/x86_64-pc-linux-gnu-library/4.2/tidyverse’

Warning in install.packages :

installation of package ‘tidyverse’ had non-zero exit status

#############################################################################

Well, to solve this i googled and got some way thereafter i tried myself. So solution is here:

1. > find("install.packages")

then i got

[1] "package:utils"

2. > sessionInfo()

then i got

R version 4.2.1 (2022-06-23)

Platform: x86_64-pc-linux-gnu (64-bit)

Running under: Ubuntu 22.10

Matrix products: default

BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.1

LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.1

locale:

[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=de_DE.UTF-8

[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8

[7] LC_PAPER=de_DE.UTF-8 LC_NAME=C LC_ADDRESS=C

[10] LC_TELEPHONE=C LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

attached base packages:

[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):

[1] compiler_4.2.1 tools_4.2.1

Unfortunately, tidyverse is not installed.

Well, i again i googled.

3. > utils::install.packages("tidyverse")

I got this error

ERROR: configuration failed for package ‘curl’