Please enable JavaScript in your browser to get the full Trove experience.
All editions
The space of memory : language and culture of Portocannone, a small Albanian village in Italy : Gjaku Jone? i She?prishur (The Albanian Diaspora) / by Elsa Musacchio and Barbara Gabriella R edited by Giada Mangiardi
alt="The space of memory : language and culture of Portocannone, a small Albanian village in Italy : Gjaku Jone? i She?prishur (The Albanian Diaspora) / by Elsa Musacchio and Barbara Gabriella R edited by Giada Mangiardi"
onload="if (this.naturalWidth > 10 && this.naturalHeight > 10)$(this).removeClass('hide');"class="hide coverArtImage"
10 && this.naturalHeight > 10)$(this).removeClass('hide');"class="hide"/>
10 && this.naturalHeight > 10)$(this).removeClass('hide');"class="hide"/>
&&In 2000, two linguistic endangerment experts, Nettle and Romaine, estimated that among the world's 6000 languages, only about 600 could be considered safe today. Arbe?resh, the language that Barbara Renzi and Elsa Musacchio present to the readers in this volume, is one of those languages in danger of dying, recognized as such in UNESCO's Atlas of the World's Languages in Danger (2010). The great value of this volume is not limited to the preservation and analysis of cultural heritage through ...
of this work
Refine your editions:
User activity
Be the first to add this to a list
user comments or reviews for this workTraumatic Brain Injuries: Memory
Impaired memory is one of the
universal problems of people with head injury. All of my patients have complained about
memory problems following their injury. Most people think of memory as being &good or
bad.& As we get older, we recognize that our memory isn't as good as it used to be.
It's more complicated than that. There are several different types of memory . Let’s
take a look at them.
TYPES OF MEMORY
First, we’ll look at the
different types of memory. For example, we all have memory for music. We can be listening
to a song on the radio and have a very distinct feeling associated with that music. The
brain processes music and puts that information in one part of the brain. We also have
memories for taste and smell. We know the taste of chocolate. We know the smell of burning
rubber. We have memories for the things we feel (physical). We can remember the difference
between the feel of silk and the feel of sand paper. Each type of memory has a different
site in the brain. Two of the more important types of memory are vision and hearing (in
this case, words). Visual things are the things we see, such as a familiar place or where
we’ve left our car. We also have memory for language, including things that we've
heard or read (things we've read we translate into language). Verbal information is stored
in the left hemisphere with visual information stored in the right hemisphere of the
IMMEDIATE MEMORY
Information going into the brain
is processed at several stages. I'm going to simplify at this point and discuss what I
call immediate memory. Immediate memory really doesn't last very long--perhaps minutes.
When do you use immediate memory? When you call information for a phone number, the
operator will tell you a seven digit number. If you're pretty good, you can remember those
seven digits long enough to dial the phone. That's immediate memory--information that is
briefly saved. With people who have a head injury, immediate memory can be
&good& or it can be &bad.& The problem for most head-injured people,
however, is with short-term memory.
SHORT-TERM MEMORY
There's some variation in how
people define short-term memory. I define it as the ability to remember something after 30
minutes. In a head injury, someone's immediate memory may be good, yet they may still have
problems with short-term memory. For example, a nurse in the hospital asked a head-injured
patient to get up and take a shower and get breakfast. The patient said that he would, but
the nurse came back 30 minutes later and the patient was still sitting in bed. When the
nurse asked him why he didn't get up and take a shower, he said that the nurse never told
him. So immediate memory is something you quickly &spit back&, but the problem
rests more with short-term memory. For example, someone may tell you to go to the store
and get some milk, some eggs, a newspaper, and some dish soap. By the time you get to the
store, all that you remember is the milk. In head injury, impaired short-term memory is a
very significant problem.
LONG-TERM MEMORY
Long-term memory is information
that we recall after a day, two weeks, or ten years. For most head-injured people, their
long-term memory tends to be good. One patient told me &I can tell you what
happened 10 years a I just can't tell you what happened 10 minutes
ago.& After you get a head injury, short-term memory isn't working, so
information has a hard time getting to long-term memory. For example, head-injured people
may double or triple their usual study time in preparing for a test the next day. By the
time they get to the exam, they are completely blank on the material. People with head
injuries have also told me &you know, time just seems to fly by.& The
little events of the day are sometimes forgotten, making life &fly by& when you
look back at events that have happened since the injury.
LOST MEMORIES--TWO TYPES
OF AMNESIA
Before we go on with memory,
let's talk about two common things that happen with people with head injuries: retrograde
and anterior grade amnesia. Amnesia means you lost a memory that you once had. It's as if
someone has erased part of your past. Retrograde amnesia means you have lost memories for
events PRIOR to the accident. For some people, retrograde amnesia can cover just a minute
or even a few seconds. In other words, they'll recall the car coming right at them but are
unable to recall the moment of impact. For other people, retrograde amnesia may affect
longer periods of time. The last three or four hours prior to the accident are gone. I had
one individual who had lost the last year of his life. As people get better from their
head injuries, long-term memories tend to return. However, memories tend to return like
piece these bits and pieces return in random order. In general, the
smaller the degree of retrograde amnesia, the less significant the head injury.
Another form of memory loss is
called anterior grade amnesia. In this case, events FOLLOWING the accident have been
erased. A good part of that is due to the brain injury itself. Complex systems in the
brain are injured. The chemical balance in the brain is upset. As brain chemistry
normalizes and brain systems begin working, memory also starts to work. I've had patients
who have spent several months in the hospital but are only able to recall the last to two
to three weeks of their stay.
WHY IS MEMORY IMPAIRED?
Why doesn't my short-term memory
work? Well, let's quickly review how the brain works. We know the information flows in
through the middle of our brain and branches out like a tree. Before that information goes
to different areas, it goes through a channeling/filtering system. It's almost like a mail
room--this information goes into this box, and that letter goes into that box. When the
brain is injured, these middle areas get pressed upon because of swelling (pressure pushes
down on the brain). The middle sections of the brain are also resting on the bone of the
skull. Because of forward and backward movement of the brain in an accident, they get
sheered or torn. A problem develops when there is a large flow of information coming in
which the brain can't process, or when information is not being sent to the right place.
So the mail room of the brain is not doing its job.
There is also a second type of
memory problem. Once information is stored in the brain, the brain has a hard time finding
it. For example, you saw a movie but you can't recall the name of the actor in the movie.
You can visualize who the actor is, but can't come up with his name. People typically
describe a &tip of the tongue& type of thing--&I know what I want
to say but I just can't get it out&. Its almost as if the brain is saying,
&searching, searching& and not finding. Several minutes later, it just comes to
you. So think of it as a library in some sense. If I take a book on history and I just put
it anywhere in the library, I'm going to have to search that whole library to get that one
book. So there are basically two kinds of memory problems: storage problems and retrieval
WHAT CAN I DO TO IMPROVE
MY MEMORY?
Work with a Specialist in
Memory--One of the most important things is to get help from people who
specialize in head injury. Every head injury program has a specialist who teaches memory
strategies. In most cases, this is a Speech Therapist (they don't just help people who
have slurred speech). In the Neuro-Recovery program, our Speech Therapist teaches 15
different memory strategies and helps you to pick the 2 or 3 that work best for you. There
is often a fair amount of testing in order to figure out the best memory strategy for each
head-injured person. For some people, one type of memory may be impaired (verbal recall)
but another type be intact (remembering visual information). If I know that my verbal
memory is not very good, I write things down and encourage visual memory systems to work.
Specialists can help you pick out the best memory strategies to help you. Once you find an
effective strategy, keep working on it. Think of memory like a muscle. The more you use
it, the stronger it gets.
Get Organized--We
learn better if we are organized. Many people have told me that, prior to their head
injury, they had incredibly messy desks with papers all over. But if someone came in and
said, &I need this particular paper&, they could pull it out of a big
pile and say &here it is.& After a head injury, though, the ability to
organize gets really messed up. One symptom of not being organized is when someone says,
&I've started 50 projects and haven't finished one of them.& If you
organize information, it tends to help you recall it. For example, if you are constantly
losing your car keys or constantly forgetting where you put your wallet, there's one
simple technique to use. Put things in the same place. Always put your car keys in one
spot on the dresser. Always put your purse in one spot in the house and nowhere else.
Being organized helps your memory and you will be less likely to lose things.
Break It Down--Another
thing that we can do to help memory is to break it into small bits. If you have something
really tough to learn, try to break it down into small bits and then learn each one little
bit at a time. Some people call this && you are memorizing little
&chunks& of information. For example, your brand new VCR has a remote control
with 50 buttons on it. Reading the entire manual in one sitting to learn what all of the
50 buttons do is very hard. So, learn one function and then play with that feature for
awhile. Once you've learned that, go on to the next button. We've been using this
technique for years to learn simple information like a phone number. The wonderful folks
at Bell Labs (they invented the phone) figured out that people will learn a 7 digit phone
number if you group 3 digits together and then group 4 digits together (a
&chunk& of 3 numbers and a &chunk& of 4 numbers).
Using Association--
Association is really important for retrieving important information. For example, you are
taking a literature course and you need to remember a famous essayist--Francis Bacon. You
might associate the image of a piece of bacon with the name of this person. So if you're
trying to think of this explorer, an image of a piece of bacon will come to you. This
approach is particularly helpful with learning names. Remembering names is a difficult
task for most it is especially hard for most people with a head
Get a Daily Planner--Probably
one of the best things you can do to help your memory is to use a daily planner. This
brings up two important points:
The First Rule of Memory--write everything down in
one spot (your daily planner).
The Second Rule of Memory--write it down when it's
fresh in your mind.
For example, you go to your
doctor's office and you are asked to return for another appointment. Many people have a
calendar stuck on their refrigerator or on a wall at home. By the time you get back home,
you've forgotten the date or lost the appointment card. Next time, bring a planner to the
doctor's office and write your appointment in it just after the doctor tells you the date.
Get a medium size planner or something called an organizer. Don't get something that's too
small--you're going to be doing a lot of writing. Write complete notes! Some people make
notes so short that they later can't figure out what the note means.
Make A &To Do&
List--In addition to a planner, make a &to do& list. For example, you
may have a number of chores to do around the house but none of them in any particular
order. What you can do is get a small pad of paper and write down the things that they
have to do. Once you have this list, decide which task to do first, second, third, and so
on. This will work if your list doesn't get too long. If the list gets too long, you're
going to run into problems.
Make a &Modified To
Do& List--I commonly hear the same problem, &I've got 50 projects
going but I haven't finished any of them.& This is a combined problem of memory
deficits and organizational deficits. One solution is to buy a small dry-erase board and
put it up in the home (or office). On the board, you are only allowed to list five items
on the &To Do& list. You cannot add another item to the board until you have
completed one of the items already on the board. Make a &Modified To Do List&
and put it somewhere in your house where the whole family can see it. Family can also
offer suggestions to help you to get projects organized. This will in turn help family
members get a better understanding of what the head-injured person has to deal with. When
you get organized and use the Planner/To Do List, you'll feel better about yourself
because you will be getting things accomplished.
THINGS THAT CAN AFFECT
Fatigue and Sleep--Memory
can be affected by a number of things. Some of these influences can affect even people
without a head injury. If you do have a head injury, these effects are multiplied. For
example, if you're tired, your memory tends to be poor. If you have to learn something
really important, it's best to learn it when you’re fresh. Generally, most people
with head injuries learn better in the morning than in the late afternoon. So if you've
got something really important (a test for school), studying the night before the exam may
not be very useful. The best time to study is in the morning. In addition, people with
head injuries commonly have sleep disorders. Its very important that you talk with your
doctor about getting your sleep pattern back to normal. If you wake up tired, you're going
to make memory problems worse. (See the sections of this book on Sleep Disorders and
Strong Emotions--Very
emotional situations will distort your memory. If someone comes into a bank and waves a
gun at you, this will create a very emotional situation (intense fear). Even though there
are five people looking at the same person, you will get five different descriptions of
the robber (that's one reason why they have cameras in banks). Being afraid, mad, or
anxious will alter your memories. What should you do in emotional situations? You can't
always control your emotions, but there are situations where you will need to recall
important information. For example, going to a doctor's office can be a very emotional
experience (many people are afraid or anxious). Most head-injured people find this a very
stressful situation. One approach is to bring a family member or friend along. Another
approach is to tell your doctor of your memory problems and ask the doctor if you can
write down important information.
I have included a Memory Sheet
for Doctors’ Appointments. When you get to the doctor's office, you may be so nervous
that you forget to tell your doctor ALL of your problems. The next time you see your
doctor, you may recall some of the old problems that you forgot on the first visit. A
common complaint of doctors who work with head-injured patients is, &Every time I
see this person, I hear a new problem.& Using the sheet of paper at the back of
this book can help you organize your thoughts. Doctors deal better with information when
it is presented in a quick and organized form. For each problem, I left a little space for
you to write down what the doctor recommends. In this way, you can bring your memory sheet
to future appointments and keep track of your progress. For example, maybe you're not
sleeping and one medication approach is being tried. If that approach doesn't work or has
a limited effectiveness, you will want to keep track of that and bring it up the next time
you see your doctor. If you don't write these problems down and monitor them, you'll be
less likely to make progress with them.
Certain types of medications
(pain killers, for example) can affect memory. In the hospital, people can be on fairly
significant amounts of pain killers because they have broken bones or other injuries. Some
medications can make you tired, which in turn makes your memory worse. It is important to
have an ongoing dialogue with your doctor about your medications. People have told me,
&I'd rather be in a little bit pain so I can be more alert.& Sometimes
the goal of pain medication is not to get rid of all the pain, but to control it. Talk
with your doctor about your medications and the possible effect of medications on memory.
|| How the Brain is Hurt
COPING WITH COMMON
Memory || Headaches ||
|| Fatigue ||
BRAIN INJURY SURVIVAL GUIDE
By Dr. Glen Johnson, Clinical Neuropsychologist
5123 North Royal Drive || Traverse City, MI 49684
Phone: 231-929-7358 || Email:
Copyright &2010 Dr.
Glen Johnson. All Rights Reserved.The details of R’s memory management are not documented in a single place. Most of the information in this chapter was gleaned from a close reading of the documentation (particularly ?Memory and ?gc), the
section of R-exts, and the
section of R-ints. The rest I figured out by reading the C source code, performing small experiments, and asking questions on R-devel. Any mistakes are entirely mine.
Object size
To understand memory usage in R, we will start with pryr::object_size(). This function tells you how many bytes of memory an object occupies:
library(pryr)
object_size(1:10)
object_size(mean)
object_size(mtcars)
## 6.74 kB
(This function is better than the built-in object.size() because it accounts for shared elements within an object and includes the size of environments.)
Something interesting occurs if we use object_size() to systematically explore the size of an integer vector. The code below computes and plots the memory usage of integer vectors ranging in length from 0 to 50 elements. You might expect that the size of an empty vector would be zero and that memory usage would grow proportionately with length. Neither of those things are true!
sizes &- sapply(0:50, function(n) object_size(seq_len(n)))
plot(0:50, sizes, xlab = &Length&, ylab = &Size (bytes)&,
type = &s&)
This isn’t just an artefact of integer vectors. Every length 0 vector occupies 40 bytes of memory:
object_size(numeric())
object_size(logical())
object_size(raw())
object_size(list())
Those 40 bytes are used to store four components possessed by every object in R:
Object metadata (4 bytes). These metadata store the base type (e.g. integer) and information used for debugging and memory management.
Two pointers: one to the next object in memory and one to the previous object (2 * 8 bytes). This doubly-linked list makes it easy for internal R code to loop through every object in memory.
A pointer to the attributes (8 bytes).
All vectors have three additional components:
The length of the vector (4 bytes). By using only 4 bytes, you might expect that R could only support vectors up to 24 × 8 - 1 (231, about two billion) elements. But in R 3.0.0 and later, you can actually have vectors up to 252 elements.
to see how support for long vectors was added without having to change the size of this field.
The “true” length of the vector (4 bytes). This is basically never used, except when the object is the hash table used for an environment. In that case, the true length represents the allocated space, and the length represents the space currently used.
The data (?? bytes). An empty vector has 0 bytes of data. Numeric vectors occupy 8 bytes for every element, integer vectors 4, and complex vectors 16.
If you’re keeping count you’ll notice that this only adds up to 36 bytes. The remaining 4 bytes are used for padding so that each component starts on an 8 byte (= 64-bit) boundary. Most cpu architectures require pointers to be aligned in this way, and even if they don’t require it, accessing non-aligned pointers tends to be rather slow. (If you’re interested, you can read more about it in .)
This explains the intercept on the graph. But why does the memory size grow irregularly? To understand why, you need to know a little bit about how R requests memory from the operating system. Requesting memory (with malloc()) is a relatively expensive operation. Having to request memory every time a small vector is created would slow R down considerably. Instead, R asks for a big block of memory and then manages that block itself. This block is called the small vector pool and is used for vectors less than 128 bytes long. For efficiency and simplicity, it only allocates vectors that are 8, 16, 32, 48, 64, or 128 bytes long. If we adjust our previous plot to remove the 40 bytes of overhead, we can see that those values correspond to the jumps in memory use.
plot(0:50, sizes - 40, xlab = &Length&,
ylab = &Bytes excluding overhead&, type = &n&)
abline(h = 0, col = &grey80&)
abline(h = c(8, 16, 32, 48, 64, 128), col = &grey80&)
abline(a = 0, b = 4, col = &grey90&, lwd = 4)
lines(sizes - 40, type = &s&)
Beyond 128 bytes, it no longer makes sense for R to manage vectors. After all, allocating big chunks of memory is something that operating systems are very good at. Beyond 128 bytes, R will ask for memory in multiples of 8 bytes. This ensures good alignment.
A subtlety of the size of an object is that components can be shared across multiple objects. For example, look at the following code:
x &- 1:1e6
object_size(x)
y &- list(x, x, x)
object_size(y)
y isn’t three times as big as x because R is smart enough to not copy x instead it just points to the existing x.
It’s misleading to look at the sizes of x and y individually. If you want to know how much space they take up together, you have to supply them to the same object_size() call:
object_size(x, y)
In this case, x and y together take up the same amount of space as y alone. This is not always the case. If there are no shared components, as in the following example, then you can add up the sizes of individual components to find out the total size:
x1 &- 1:1e6
y1 &- list(1:1e6, 1:1e6, 1:1e6)
object_size(x1)
object_size(y1)
object_size(x1, y1)
object_size(x1) + object_size(y1) == object_size(x1, y1)
## [1] TRUE
The same issue also comes up with strings, because R has a global string pool. This means that each unique string is only stored in one place, and therefore character vectors take up less memory than you might expect:
object_size(&banana&)
object_size(rep(&banana&, 10))
Repeat the analysis above for numeric, logical, and complex vectors.
If a data frame has one million rows, and three variables (two numeric, and one integer), how much space will it take up? Work it out from theory, then verify your work by creating a data frame and measuring its size.
Compare the sizes of the elements in the following two lists. Each contains basically the same data, but one contains vectors of small strings while the other contains a single long string.
vec &- lapply(0:50, function(i) c(&ba&, rep(&na&, i)))
str &- lapply(vec, paste0, collapse = &&)
Which takes up more memory: a factor (x) or the equivalent character vector (as.character(x))? Why?
Explain the difference in size between 1:5 and list(1:5).
Memory usage and garbage collection
While object_size() tells you the size of a single object, pryr::mem_used() tells you the total size of all objects in memory:
library(pryr)
mem_used()
This number won’t agree with the amount of memory reported by your operating system for a number of reasons:
It only includes objects created by R, not the R interpreter itself.
Both R and the operating system are lazy: they won’t reclaim memory until it’s actually needed. R might be holding on to memory because the OS hasn’t yet asked for it back.
R counts the memory occupied by objects but there may be gaps due to deleted objects. This problem is known as memory fragmentation.
mem_change() builds on top of mem_used() to tell you how memory changes during code execution. Positive numbers represent an increase in the memory used by R, and negative numbers represent a decrease.
# Need about 4 mb to store 1 million integers
mem_change(x &- 1:1e6)
## 4.01 MB
# We get that memory back when we delete it
mem_change(rm(x))
Even operations that don’t do anything use up a little memory. This is because R is tracking the history of everything you do. You can ignore anything smaller than a couple kB.
mem_change(NULL)
mem_change(NULL)
In some languages, you have to explicitly delete unused objects for their memory to be returned. R uses an alternative approach: garbage collection (or GC for short). GC automatically releases memory when an object is no longer used. It does this by tracking how many names point to each object, and when there are no names pointing to an object, it deletes that object.
# Create a big object
mem_change(x &- 1:1e6)
# Also point to 1:1e6 from y
mem_change(y &- x)
## 1.36 kB
# Remove x, no memory freed because y is still pointing to it
mem_change(rm(x))
## 1.25 kB
# Now nothing points to it and the memory can be freed
mem_change(rm(y))
Despite what you might have read elsewhere, there’s never any need to call gc() yourself. R will automatically run garbage collection whenever if you want to see when that is, call gcinfo(TRUE). The only reason you might want to call gc() is to ask R to return memory to the operating system. However, even that might not have any effect: older versions of Windows had no way for a program to return memory to the OS.
GC takes care of releasing objects that are no longer used. However, you do need to be aware of possible memory leaks. A memory leak occurs when you keep pointing to an object without realising it. In R, the two main causes of memory leaks are formulas and closures because they both capture the enclosing environment. The following code illustrates the problem. In f1(), 1:1e6 is only referenced inside the function, so when the function completes the memory is returned and the net memory change is 0. f2() and f3() both return objects that capture environments, so that x is not freed when the function completes.
f1 &- function() {
x &- 1:1e6
mem_change(x &- f1())
## 1.26 kB
object_size(x)
f2 &- function() {
x &- 1:1e6
mem_change(y &- f2())
object_size(y)
f3 &- function() {
x &- 1:1e6
function() 10
mem_change(z &- f3())
object_size(z)
## 4.01 MB
Memory profiling with lineprof
mem_change() captures the net change in memory when running a block of code. Sometimes, however, we may want to measure incremental change. One way to do this is to use memory profiling to capture usage every few milliseconds. This functionality is provided by utils::Rprof() but it doesn’t provide a very useful display of the results. Instead we’ll use the
package. It is powered by Rprof(), but displays the results in a more informative manner.
To demonstrate lineprof, we’re going to explore a bare-bones implementation of read.delim() with only three arguments:
read_delim &- function(file, header = TRUE, sep = &,&) {
# Determine number of fields by reading first line
first &- scan(file, what = character(1), nlines = 1,
sep = sep, quiet = TRUE)
p &- length(first)
# Load all fields as character vectors
all &- scan(file, what = as.list(rep(&character&, p)),
sep = sep, skip = if (header) 1 else 0, quiet = TRUE)
# Convert from strings to appropriate types (never to factors)
all[] &- lapply(all, type.convert, as.is = TRUE)
# Set column names
if (header) {
names(all) &- first
names(all) &- paste0(&V&, seq_along(all))
# Convert list into data frame
as.data.frame(all)
We’ll also create a sample csv file:
library(ggplot2)
write.csv(diamonds, &diamonds.csv&, row.names = FALSE)
Using lineprof is straightforward. source() the code, apply lineprof() to an expression, then use shine() to view the results. Note that you must use source() to load the code. This is because lineprof uses srcrefs to match up the code and run times. The needed srcrefs are only created when you load code from disk.
library(lineprof)
source(&code/read-delim.R&)
prof &- lineprof(read_delim(&diamonds.csv&))
shine(prof)
shine() will also open a new web page (or if you’re using RStudio, a new pane) that shows your source code annotated with information about memory usage. shine() starts a shiny app which will “block” your R session. To exit, press escape or ctrl + break.
Next to the source code, four columns provide details about the performance of the code:
t, the time (in seconds) spent on that line of code (explained in ).
a, the memory (in megabytes) allocated by that line of code.
r, the memory (in megabytes) released by that line of code. While memory allocation is deterministic, memory release is stochastic: it depends on when the GC was run. This means that memory release only tells you that the memory released was no longer needed before this line.
d, the number of vector duplications that occurred. A vector duplication occurs when R copies a vector as a result of its copy on modify semantics.
You can hover over any of the bars to get the exact numbers. In this example, looking at the allocations tells us most of the story:
scan() allocates about 2.5 MB of memory, which is very close to the 2.8 MB of space that the file occupies on disk. You wouldn’t expect the two numbers to be identical because R doesn’t need to store the commas and because the global string pool will save some memory.
Converting the columns allocates another 0.6 MB of memory. You’d also expect this step to free some memory because we’ve converted string columns into integer and numeric columns (which occupy less space), but we can’t see those releases because GC hasn’t been triggered yet.
Finally, calling as.data.frame() on a list allocates about 1.6 megabytes of memory and performs over 600 duplications. This is because as.data.frame() isn’t terribly efficient and ends up copying the input multiple times. We’ll discuss duplication more in the next section.
There are two downsides to profiling:
read_delim() only takes around half a second, but profiling can, at best, capture memory usage every 1 ms. This means we’ll only get about 500 samples.
Since GC is lazy, we can never tell exactly when memory is no longer needed.
You can work around both problems by using torture = TRUE, which forces R to run GC after every allocation (see gctorture() for more details). This helps with both problems because memory is freed as soon as possible, and R runs 10–100x slower. This effectively makes the resolution of the timer greater, so that you can see smaller allocations and exactly when memory is no longer needed.
When the input is a list, we can make a more efficient as.data.frame() by using special knowledge. A data frame is a list with class data.frame and row.names attribute. row.names is either a character vector or vector of sequential integers, stored in a special format created by .set_row_names(). This leads to an alternative as.data.frame():
to_df &- function(x) {
class(x) &- &data.frame&
attr(x, &row.names&) &- .set_row_names(length(x[[1]]))
What impact does this function have on read_delim()? What are the downsides of this function?
Line profile the following function with torture = TRUE. What is surprising? Read the source code of rm() to figure out what’s going on.
f &- function(n = 1e5) {
x &- rep(1, n)
Modification in place
What happens to x in the following code?
x[5] &- 10
There are two possibilities:
R modifies x in place.
R makes a copy of x to a new location, modifies the copy, and then uses the name x to point to the new location.
It turns out that R can do either depending on the circumstances. In the example above, it will modify in place. But if another variable also points to x, then R will copy it to a new location. To explore what’s going on in greater detail, we use two tools from the pryr package. Given the name of a variable, address() will tell us the variable’s location in memory and refs() will tell us how many names point to that location.
library(pryr)
c(address(x), refs(x))
# [1] &0x& &1&
c(address(y), refs(y))
# [1] &0x& &2&
(Note that if you’re using RStudio, refs() will always return 2: the environment browser makes a reference to every object you create on the command line.)
refs() is only an estimate. It can only distinguish between one and more than one reference (future versions of R might do better). This means that refs() returns 2 in both of the following cases:
# Should really be 1, because we've deleted y
# Should really be 3
When refs(x) is 1, modification will occur in place. When refs(x) is 2, R will make a copy (this ensures that other pointers to the object remain unaffected). Note that in the following example, y keeps pointing to the same location while x changes.
c(address(x), address(y))
## [1] &0x6a52c98& &0x6a52c98&
x[5] &- 6L
c(address(x), address(y))
## [1] &0x6adbd18& &0x6a52c98&
Another useful function is tracemem(). It prints a message every time the traced object is copied:
# Prints the current memory location of the object
tracemem(x)
# [1] &&0x7feeaaa1c6b8&&
x[5] &- 6L
# Prints where it has moved from and to
x[5] &- 6L
# tracemem[0x7feeaaa1c6b8 -& 0x7feeaaa1c768]:
For interactive use, tracemem() is slightly more useful than refs(), but because it just prints a message, it’s harder to program with. I don’t use it in this book because it interacts poorly with , the tool I use to interleave text and code.
Non-primitive functions that touch the object always increment the ref count. Primitive functions usually don’t. (The reasons are a little complicated, but see the R-devel thread .)
# Touching the object forces an increment
f &- function(x) x
{x &- 1:10; f(x); refs(x)}
# Sum is primitive, so no increment
{x &- 1:10; sum(x); refs(x)}
# f() and g() never evaluate x, so refs don't increment
f &- function(x) 10
g &- function(x) substitute(x)
{x &- 1:10; f(x); refs(x)}
{x &- 1:10; g(x); refs(x)}
Generally, provided that the object is not referred to elsewhere, any primitive replacement function will modify in place. This includes [[&-, [&-, @&-, $&-, attr&-, attributes&-, class&-, dim&-, dimnames&-, names&-, and levels&-. To be precise, all non-primitive functions increment refs, but a primitive function may be written in such a way that it doesn’t. The rules are sufficiently complicated that there’s little point in trying to memorise them. Instead, you should approach the problem practically by using refs() and address() to figure out when objects are being copied.
While determining that copies are being made is not hard, preventing such behaviour is. If you find yourself resorting to exotic tricks to avoid copies, it may be time to rewrite your function in C++, as described in .
For loops in R have a reputation for being slow. Often that slowness is because you’re modifying a copy instead of modifying in place. Consider the following code. It subtracts the median from each column of a large data frame:
x &- data.frame(matrix(runif(100 * 1e4), ncol = 100))
medians &- vapply(x, median, numeric(1))
for(i in seq_along(medians)) {
x[, i] &- x[, i] - medians[i]
You may be surprised to realise that every iteration of the loop copies the data frame. We can see that more clearly by using address() and refs() for a small sample of the loop:
for(i in 1:5) {
x[, i] &- x[, i] - medians[i]
print(c(address(x), refs(x)))
## [1] &0x303a4a0& &2&
## [1] &0x315cec0& &2&
## [1] &0x3d56de0& &2&
## [1] &0x3b80380& &2&
## [1] &0x6bcaa00& &2&
For each iteration, x is moved to a new location so refs(x) is always 2. This occurs because [&-.data.frame is not a primitive function, so it always increments the refs. We can make the function substantially more efficient by using a list instead of a data frame. Modifying a list uses primitive functions, so the refs are not incremented and all modifications occur in place:
y &- as.list(x)
for(i in 1:5) {
y[[i]] &- y[[i]] - medians[i]
print(c(address(y), refs(y)))
## [1] &0x2388b50& &1&
## [1] &0x2388b50& &1&
## [1] &0x2388b50& &1&
## [1] &0x2388b50& &1&
## [1] &0x2388b50& &1&
This behaviour was substantially more problematic prior to R 3.1.0, because every copy of the data frame was a deep copy. This made the motivating example take around 5 s, compared to 0.01 s today.
The code below makes one duplication. Where does it occur and why? (Hint: look at refs(y).)
y &- as.list(x)
for(i in seq_along(medians)) {
y[[i]] &- y[[i]] - medians[i]
The implementation of as.data.frame() in the previous section has one big downside. What is it and how could you avoid it?